Convergent Sequences of Dense Graphs I: 
Subgraph Frequencies, Metric Properties and Testing 



C. Borgs", J.T. Chayes", L. Lovasz^t V.T. Sos^J 
K. Vesztergombi"^ 

"'Microsoft Research, One Microsoft Way, Redmond, WA 98052, USA 
^Alfred Renyi Institute of Mathematics, FOB 127, H-1364 Budapest, Hungary 
^Eotvos Lordnd University, Pdzmdny Peter Setani 1/C, H- 11 17 Budapest, Hungary 

May 2006 
(revised January 2007) 



Abstract 

We consider sequences of graphs and define various notions of convergence related 
to these sequences: "left convergence" defined in terms of the densities of homomorphisms 
from small graphs into G„; "right convergence" defined in terms of the densities of homo- 
morphisms from Gn into small graphs; and convergence in a suitably defined metric. 

In Part I of this series, we show that left convergence is equivalent to convergence in 
metric, both for simple graphs Gn, and for graphs G„ with nodeweights and edgeweights. 
One of the main steps here is the introduction of a cut-distance comparing graphs, not 
necessarily of the same size. We also show how these notions of convergence provide natural 
formulations of Szemeredi partitions, sampling and testing of large graphs. 
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1 Introduction 



In this and accompanying papers, we define a natural notion of convergence of a sequence of 
graphs, and show that other useful notions of convergence are equivalent to it. We are motivated 
by the fact that, in many subfields of mathematics, computer science and physics, one studies 
properties of very large graphs, or properties of graph sequences that grow beyond all limits. Let 
us give a few examples: 

Random Networks: There is a large literature of graph models of the Internet, the WWW, 
and other so-called " scale-free" technological networks, first modeled in this context by Barabasi 
and Albert [7] . Among the technological networks that are modeled are the graph of computers 
and physical links between them, the graph of so-called Autonomous Systems such as Internet 
Service Providers, the graph of webpages with hyperlinks, etc. These graphs are often similar 
to various graphs of social networks: acquaintances, co-publications, the spreading of certain 
diseases, etc. 

These networks are formed by random processes, but their properties are quite different from 
the traditional Erdos-Renyi random graphs: their degree distribution has a "heavy tail", they 
tend to be clustered, the neighborhoods of their nodes are denser than the average edge density, 
etc. Several models of random scale-free graphs have been proposed and studied. For rigorous 
work, see [TTl [T3] for undirected models, [26] for "copying models" , [10] for a directed model, 
[9] for the spread of viruses on these networks, and [I2] for a survey of rigorous work with more 
complete references. 

Quasirandom Graphs: Quasirandom (also called pseudorandom) graphs were introduced by 
Thomason [22] and Chung, Graham and Wilson [20]. These graph sequences can be deterministic, 
but have many properties of true random graphs. A nice example is the sequence of Paley graphs 
(quadratic residue graphs). These graphs are remarkably similar to a random graph with edge- 
probability 1/2 on the same number of nodes in many ways. The most relevant for us is that they 
contain (asymptotically) the same number of copies of each fixed graph F as the random graph — 
this is one of the many equivalent ways to define quasirandom graphs. Many other questions in 
graph theory, in particular in extremal graph theory, also involve asymptotic counting of small 
graphs. 

Property Testing of Large Graphs: Say we are given a large graph and we want to determine 
certain numerical parameters, e.g., the edge density, of that graph by sampling a bounded number 
of nodes. Or perhaps we want to determine whether the large graph has a given property, 
e.g., is it 3-colorable? In particular, which parameters can be accurately estimated and which 
properties can be tested with high probability by looking only at subgraphs on small randomly 
chosen subsets of the nodes? A precise definition of property testing was given by Goldreich, 
Goldwasser and Ron [23], who also proved several fundamental results about this problem. 
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Statistical Mechanics: Many models in physics are described by a weighted coloring of some 
large graph G. The graph G typically represents underlying geometric structure of the model 
under consideration, e.g. a crystal lattice and its nearest neighbor structure, while the color 
of a given node represents the local state. In the simplest case of two colors, the two colors 
can represent quantities like the two possible orientations of a spin variable, or the presence or 
absence of a molecule at a given position. The interactions between different local states can then 
be described by a weighted "interaction graph" H, with smaller edgeweights corresponding to 
weaker interactions, and larger edgeweights representing stronger interactions. In this context, 
the weighted number of colorings represents the so-called partition function of the model. 

Combinatorial Optimization: Many optimization problems can be described as weighted col- 
oring problems. A simple example is the max-cut problem, where our task is to find the maximal 
cut in a large graph G. If we consider a coloring of G with two colors, 1 and 2, and weight a 
coloring by the number of edges with two differently colored endnodes, then the maximum cut 
is just given by the maximum weight coloring. 

In this and two accompanying papers we develop a theory of convergence of graph sequences, 
which works best in two extreme cases: dense graphs (the subject of this paper and [16]) and 
graphs with bounded degree (the subject of ^7\). Convergence of graph sequences was defined 
by Benjamini and Schramm [8] for graphs with bounded degree, and by the authors of this paper 
[13] for dense graphs. 

Our general setup will be the following. We have a "large" graph G with node set V{G) 
and edge set E{G). There are (at least) two ways of studying G using homomorphisms. First, 
we can count the number of copies of various "small" graphs F in G, more precisely, we count 
the number of homomorphisms from F to G; this way of looking at G allows us to treat many 
problems in, e.g., extremal graph theory. Second, we can count homomorphisms from G into 
various small graphs if; this includes many models in statistical physics and many problems on 
graph coloring. 

These two notions of probing a large graph with a small graph lead to two different notions 
of convergence of a sequence of graphs (G„): convergence from the left, corresponding to graphs 
which look more and more similar when probed with homomorphisms from small graphs into 
Gni and convergence from the right, corresponding to graph sequences whose elements look more 
and more similar when probed with homomorphism from Gn into a small graphs. 

This theory can also be viewed as a substantial generalization of the theory of quasirandom 
graphs. In fact, most of the equivalent characterizations of quasirandom graphs are immediate 
corollaries of the general theory developed here and in our companion paper [T6] . 

In this paper we study convergence from the left, both for sequences of simple graphs and 
sequences of weighted graphs, and its relations to sampling and testing. Since this paper focuses 
on convergence from the left, we'll often omit the phrase "from the left". We will also show that 
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convergence from the left is equivalent to convergence in metric for a suitable notion of distance 
between two weighted graphs. Finally, we will show that convergence from the left is equivalent to 
the property that the graphs in the sequence have asymptotically the same Szemeredi partitions. 

Convergence from the right will be the subject matter of the sequel of this paper |16j . 

Convergence in metric clearly allows for a completion by the usual abstract identification of 
Cauchy sequences of distance zero. But it turns out (Lovasz and Szegedy [27]) that the limit 
object of a convergent graph sequence has a much more natural representation in terms of a 
measurable symmetric function W : [0, 1]^ — M (we call these functions graphons). In fact, it is 
often useful to represent a finite graph G in terms of a suitable function Wq on [0, 1]^, defined 
as step function with steps of length and values and 1, see below for the precise 

definition. While the introduction of graphons requires some basic notions of measure theory, it 
will simplify many proofs in this paper. 

The organization of this paper is as follows: In the next section, we introduce our definitions: 
in addition to left- convergence, we define a suitable distance between weighted graphs, and state 
our main results for weighted graphs. In Section [3l we generalize these definitions and results 
to graphons. The following section. Section IH is devoted to sampling, and contains the proofs 
of the main results of this paper, including the equivalence of left-convergence and convergence 
in metric. Section [5] relates convergence in metric to an a priori weaker form of "convergence 
in norm" and to Szemeredi partitions, and Section [B] proves our results on testing. We close 
this paper with a section on miscellaneous results and an outlook on right-convergence. In the 
appendix, we describe a few details of proofs which are omitted in the main body of the paper. 

2 Weighted and Unweighted Graphs 
2.1 Notation 

We consider both unweighted, simple graphs and weighted graphs, where, as usual, a simple 
graph G is a graph without loops or multiple edges. We denote the node and edge set of G by 
V{G) and E{G), respectively. 

A weighted graph G is a graph with a weight = ai{G) > associated with each node and 
a weight Pij = Pij^G) G M associated with each edge ij, including possible loops with i = j. For 
convenience, we set Pij = if ij ^ E{G). We set 



aG = 5^«.(G), ||G|| 



oo 



= max 




and for S,T C. V{G), we define 




(2.1) 
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A weighted graph G is called soft-core if it is a complete graph with loops at each node, and 
every edgeweight is positive. An unweighted graph is a weighted graph where all the node- and 
edgeweights are 1. Note that edS, T) reduces to the number of edges in G with one endnode in 
S and the other in T if G is unweighted. 

Let G be a graph and k > 1. The k-fold blow-up of G is the graph G[k] obtained from G by 
replacing each node by k independent nodes, and connecting two new nodes if and only if their 
originals were connected. If G is weighted, we define G[k] to be the graph on nk nodes labeled 
by pairs iu, i G V{G), u = l,...,k, with edgeweights Piu,jv{G[k]) = and nodeweights 

aiu{G[k]) = ai{G). A related notion is the notion of splitting nodes. Here a node i with 
nodeweight ai is replaced by k nodes ii, . . . ,ik with nodeweights a^^, . . . , Oj^, adding up to Oj, 
with new edgeweights Pi^j^ = Up to a global rescaling of all nodeweights, blowing up a 
graph by a factor k is thus the same as splitting all its nodes evenly into k nodes, so that the 
new weights are equal to the old weights divided by k. 

As usual, a function from the set of simple graphs into the reals is called a simple graph 
parameter if it is invariant under relabeling of the nodes. Finally, we write G = G' if G and G' 
are isomorphic, i.e., if G' can be obtained from G by a relabeling of its nodes. 

2.2 Homomorphism Numbers and Left Convergence 

Let F and G be two simple graphs. We define hom(F, G) as the number of homomorphisms from 
F to G, i.e., the number of adjacency preserving maps V{F) V{G), and the homomorphism 
density of F in G as 

t{F,G) = j^,^^^^hom{F,G). 

The homomorphism density t{F, G) is thus the probability that a random map from V{F) to 
V{G) is a homomorphism. 

Alternatively, one might want to consider the probability tinj(-F, G) that a random injective 
map from V{F) to V{G) is adjacency preserving, or the probability tind(-F, G) that such a map 
leads to an induced subgraph. Since most maps into a large graph G are injective, there is not 
much of a difference between t[F,G) and ti^j{F,G). As for tinj(-,G) and tind(",G), they can be 
quite different even for large graphs G, but by inclusion-exclusion, the information contained in 
the two is strictly equivalent. We therefore incur no loss of generality if we restrict ourselves to 
the densities t(-,G). 

We extend the notion of homomorphism numbers to weighted graphs G by setting 

hom(F,G)= n "^W(^) n (2-2) 

<l>:V{F)^V{G)ieV{F) ij&E(F) 
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where the sum runs over all maps from V{F) to V{G), and define 



hom(F, G) 



(2.3) 



where k is the number of nodes in F. 

It seems natural to think of two graphs G and G' as similar if they have similar homomorphism 
densities. This leads to the following definition. 

Definition 2.1 Let (G„) be a sequence of weighted graphs with uniformly bounded edgeweights. 
We say that is convergent from the left, or simply convergent, if t{F, Gn) converges for any 
simple graph F. 

In |T^, the definition of convergence was restricted to sequences of graphs (Gn) with 
|y(G'„)| oo. As long as we deal with simple graphs, this is reasonable, since there are only a 
finite number of graphs with bounded size. But in this paper we also want to cover sequences of 
weighted graphs on, say, the same set of nodes, but with the node- or edgeweights converging; 
so we don't assume that the number of nodes in convergent graph sequences tends to infinity. 

A simple example of a convergent graph sequence is a sequence of random graphs for 
which t{F, Gn,p) is convergent with probability one, with t{F, Gn,p) — > p\^^^^\ as n ^ oo. Other 
examples are quasirandom graph sequences (Gn) for which t(F, Gn,p) — ^ p\^^^^\ by definition, and 
the sequence of half-graphs {Hn,n) , with Hn,n defined as the bipartite graph on [2n] with an edge 
between i and j if j > n + i (see Examples 13. ![ 13.21 and 13.31 in Section 13.11) . 

2.3 Cut-Distance 

We define a notion of distance between two graphs, which will play a central role throughout this 
paper. Among other equivalences, convergence from the left will be equivalent to convergence in 
this metric. 

To illuminate the rather technical definition, we first define a distance of two graphs G and 
G' in a special case, and then extend it in two further steps. In all these definitions, it is not 
hard to verify that the triangle inequality is satisfied. 

(1) G and G' are labeled graphs with the same set of unweighted nodes V . Several notions 
of distance appear in the literature, but for our purpose the most useful is the cut or rectangle 
distance introduced by Frieze and Kannan [22] : 




The cut distance between two labeled graphs thus measures how different two graphs are when 
considering the size of various cuts. This definition can easily be generalized to weighted graphs 




du{G, G') 



1 



eG{S,T)-eG'{S,T) . 



(2.4) 
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G and G' on the same set V , and with the same nodeweights ai = ai{G) = ai{G'): 

da(G,G') = max ^ eG(S,T) - eG'(S,T) . (2.5) 

As a motivation of this notion, consider two independent random graphs on n nodes with 
edge density 1/2. If we measure their distance, say, by the number of edges we need to change to 
get one from the other (edit distance), then their distance is very large (with large probability). 
But the theory of random graphs teaches us that these two graphs are virtually indistinguishable, 
which is reflected by the fact that their du distance is only 0{l/n) with large probability. 

There are many other ways of defining or approximating the cut distance; see Section 17.11 

(2) G and G' are unlabeled graphs with the same number of unweighted nodes. The cut- 
metric d\j{G, G') is not invariant under relabeling of the nodes of G and G' . For graphs without 
nodeweights, this can easily be cured by defining a distance 8u{G, G') as the minimum over all 
"overlays" of G and G', i.e., 

6aiG,G') =mmdaiG,G'). (2.6) 

G^G 

(3) G and G' are unlabeled graphs with different number of nodes or with weighted nodes. 
The distance notion (12. 6p does not extend in a natural way to graphs with nodeweights, since it 
would not make much sense to overlay nodes with different nodeweights, and even less sense for 
graphs with different number of nodes. 

To motivate the definition that follows, consider two graphs G and G', where G has three 
nodes with nodeweights equal to 1/3, and G' has two nodes and nodeweights 1/3 and 2/3. Here 
a natural procedure would be the following: first split the second node into two nodes of weight 
1/3 and then calculated the optimal overlay of the resulting two graphs on three nodes. 

This idea naturally leads to the following notion of "fractional overlays" of two weighted 
graphs G and G' on n and n' nodes, respectively. Let us first assume that both G and G' 
have total nodeweight 1. Viewing a{G) and a{G') as probability distributions, we then define a 
fractional overlay to be a coupling between these two distributions. More explicitly, a fractional 
overlay of G and G' is defined to be a nonnegative n x n' matrix X such that 

n' n 

^Xiu = ai{G) and ^Xj„ = a„(G"). 

u=l i=l 

We denote the set of all fractional overlays by X{G, G'). 

Let X G X{G,G'). Thinking of Xiu as the portion of node i that is mapped onto node u, 
we introduce the following " overlaid graphs" G[X] and on [n] x [n'\. in both G[X] and 

the weight of a node (i, u) G [n] x [n'] is in (^[X], the weight of an edge ((i, m), (j, v)) 
is (3ij, and in the weight of an edge ((z,u), (j, f)) is Since G[X] and G"[X'''] have 

the same nodeset, the distance (in(G[X], G'[X''']) is now well defined. Taking the minimum over 
all fractional overlays, this gives: 
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Definition 2.2 For two weighted graphs G, G' with total nodeweight ac = olg' = 1, we set 



5u{G,G') = mill dn(G[X],G'[X^]). (2.7) 

If the total nodeweight of G or G' is different from 1, we define the distance between G and G' 
by the above formulas, applied to the graphs G and G' obtained from G and G' by dividing all 
nodeweights by ac and ac', respectively. 

Fractional overlays can be understood as integer overlays of suitably blown up versions of G 
and G', at least if the entries of X are rational (otherwise, one has to take a limit of blowups). 
This observation shows that for two graphs G and G' with nodeweights one, we have 

6n{G,G')= lim 6n{G[k],G'[k']). (2.8) 

fe,fc'— >oo 
k/k' = n'/n 

Note that 5n(G, G') can be for nonisomorphic graphs G and G'; for example, 

6a{G,G[k])=0 (2.9) 

for all > 1. So 5g is only a pre-metric; but we'll call it, informally, a metric. 

Of course, the definition of 5n(G, G') also applies if G and G" have the same number of nodes, 
and it may give a value different from 6n{G, G'). The following theorem relates these two values. 

Theorem 2.3 Let Gi and G2 be two weighted graphs with edgeweights in [—1, 1] and with the 
same number of unweighted nodes. Then 

HGi, G2) < 6a{Gi, G2) < 325n(Gi, G^y/''. 

The first inequality is trivial, but the proof of the second is quite involved, and will be given 
in Section [STTl 



2.4 Szemeredi Partitions of Graphs 

The Regularity Lemma of Szemeredi is a fundamental tool in graph theory, which has a natural 
formulation in our framework, as a result about approximating large graphs by small graphs. 
Here we mostly use the so-called weak version due to Frieze and Kannan [22] . 

We need some notation: for a weighted graph G and a partition V = {Vi, . . . , Vk} of V{G), 
we define two weighted graphs G/V and Gp as follows: Let ay^ = J^xeVi^^^^)- "^^^ quotient 
graph G/V is a weighted graph on [k], with nodeweights aiiG/V) = ayjac and edgeweights 
(3ij{G/V) = while G-p is a weighted graph on V{G), with nodeweights OxiG-p) = ax{G) 

and edgeweights (3xy{Gp) = [3ij{G/V) for x E Vi and y E Vj. These two graphs have different 
number of nodes, but they are similar in the sense that 6n{G/V, Gp) = 0. 
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In our language, the Weak Regularity Lemma of Frieze and Kannan states that given a 
weighted graph G, one can find a partition V such that the graph G-p is near to the original 
graph G in the distance d\j. We call the partition V weakly e-regular if d\z\{G, G-p) < e. But for 
many purposes, all that is needed is the quotient graph G/V. Since 6o{G, Gp) < do{G, Gp) and 
5n(G, Gp) = Sa{G, G/V), the Weak Regularity Lemma also guarantees a good approximation of 
the original graph by a small weighted graph, the graph H = G/V. We summarize these facts 
in the following lemma, which is essentially a reformulation of the Weak Regularity Lemma of 
[22] in the language developed in this paper. 

Lemma 2.4 (Weak Regularity Lemma |22] ) For every e > 0, every weighted graph G has 
a partition V into at most 4^/^^ classes such that 

dniG,Gp)<e\\G\\2, (2.10) 

so, in particular, 

Sn{G,G/V)<e\\Gh. (2.11) 

In Lemma 12.41 we approximate the graph G by a small weighted graph H = G/V. If G is 
simple, or more generally, has edgeweights in [0, 1], it is possible to strengthen this by requiring 
simple graphs H. Indeed, starting from a standard strengthening of the Weak Regularity Lemma 
(Corollary 13. 41 (ii) below) to obtain a weighted graph H with nodeweights one, and then applying 
a simple randomization procedure (Lemma 14.31 below) to the edges of H to convert this graph 
into a simple graph, one gets the following lemma, see appendix for details. 

Lemma 2.5 Let e > 0, let q > 2^°^^^, and let G be a weighted graph with edgeweights in [0, 1]. 
Then there exists a simple graph H on q nodes such that 6\j{G, H) < e. 

2.5 Main Results 

2.5.1 Left Convergence Versus Convergence in Metric 

Here we state one of the main results of this paper, namely, that convergence from the left is 
equivalent to convergence in the metric (5n. 

Theorem 2.6 Let (Gn) be a sequence of weighted graphs with uniformly bounded edgeweights. 
Then {Gn) is left convergent if and only if it is a Cauchy sequence in the metric Sa- 
in fact, we have the following quantitative version. To simplify our notation, we only give this 
quantitative version for graphs with edgeweights in [—1,1]; the general case follows by simply 
scaling all edgeweights appropriately. 
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Theorem 2.7 Let Gi,G2 be weighted graphs with edgeweights in [—1,1]. 
(a) Let F he a simple graph, then 



(b) Let k > 1, and assume that \t{F,Gi) — t{F,G2)\ < 3^'^^ for every simple graph F on k 
nodes. Then 

22 

5n(Gi, G2) < 



The first part of this theorem is closely related to the "Counting Lemma" in the theory of 
Szemeredi partitions. Theorems 12.61 and 12.71 will follow from the analogous facts for graphons, 
Theorems 13.81 and 13. 7|, see Section [531 

2.5.2 Szemeredi Partitions for Graph Sequences 

Convergent graph sequences can also be characterized by the fact that (for any fixed error) they 
have Szemeredi partitions which become more and more similar. 

Theorem 2.8 Let (Gn) be a sequence of weighted graphs with nodeweights 1, edgeweights in 
[— 1, 1], and \ V{Gn)\ 00 as n —> 00. Then (Gn) is left- convergent if and only if for every e > 
we can find an integer q < 2^^/^'^ , and a sequence of partitions, (Vn) such that the following two 
conditions hold. 

(i) If \V{Gn)\ > q, then P„, is a weakly e -regular partition of Gn into q classes. 

(a) As n ^ 00, the quotient graphs Gn/Vn converge to a weighted graph on q nodes. 

Note that the graphs in (ii) have the same node set [g], so their convergence to means 
simply that corresponding nodeweights and edgeweights converge. 

Let Gn be a convergent sequence of weighted graphs obeying the assumptions of this theorem. 
For n sufficiently large, the quotient graphs Gn/Vn are then near to both the original graph Gn 
and the graph H^, implying that 6ci{Gn,Hs) < 2e whenever n is large enough. Since G„ is 
convergent, this implies by Theorem 12.61 that the graphs iJ^ form a convergent sequence as 

6^0. 



The theorem can be generalized in several directions. First, using the results of Section [5^ 
we can relax the condition on the nodeweights of Gn to require only that Gn has no dominant 
nodeweights in the sense that the maximum nodeweight of G„ divided by the total nodeweight 
aG„ goes to zero. Second, we can strengthen the statement, to obtain a sequence of partitions 
which satisfy the stronger regularity conditions of the original Szemeredi Regularity Lemma |31j . 
We leave the details to the interested reader, who will easily see how to modify the proof in 
Section 15.31 to obtain these generalizations. 
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2.5.3 Sampling 

Our above versions of Szemeredi's lemma (Lemmas 12.41 and I2.5P state tliat any grapli G can be 
well approximated by a small graph H in the 5u distance. While the proofs are constructive, it 
will be very useful to know that such a small graph can obtained by straightforward sampling. 
For simplicity, we state the results for graphs with edgeweights in [—1, 1]. For a graph G and 
positive integer n, let G(r2, G) denote the (random) induced subgraph G[S], where S is chosen 
uniformly from all subsets of V{G) of size n. 

Theorem 2.9 Let G he a weighted graph with nodeweights 1 and edgeweights in [—1,1]. Let 
k < \V{G)\. Then 

5aiG,Gik,G))<^L=. (2.12) 
V log2 k 

with probability at least 1 — e^^^/(^'°S2'=). 

In order to prove this theorem, we will need a theorem which allows us to compare samples 
from two weighted graphs on the same set of nodes. This extends a result by Alon, Fernandez de 
la Vega, Kannan and Karpinski [3l H] ; in particular, our result concerns two graphs at arbitrary 
distance d\j{Gi,G2), and also gives an improvement in the error bound. 

Theorem 2.10 Let Gi and G2 be weighted graphs on a common vertex set V , with nodeweights 
one and edgeweights in [—1,1]. Let k < \V\. If S is chosen uniformly from all subsets of V of 
size k, then 

dn{G^[SlG2[S]) - du{Gi,G2)\ < (2.13) 
with probability at least 1 — 2e~^/^. 

2.5.4 Testing 

The above theorem allows us to prove several results for testing graph parameters and graph 
properties in a straightforward way. 

In this paper, we only consider parameter testing. We may want to determine some parameter 
of G. For example, what is the edge density? Or how large is the density of the maximum cut? 
Of course, we'll not be able to determine the exact value of the parameter; the best we can 
hope for is that if we take a sufficiently large sample, we can find the approximate value of the 
parameter with large probability. 

Definition 2.11 A graph parameter / is testable if for every 5 > there is a positive integer k 
such that if G is a graph with at least k nodes, then from the random subgraph G(/i;, G) we can 
compute an estimate f of f such that 

FT{\f{G)-f\>e)<e. 
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It is an easy observation that we can always use / = f{G{k, G)). 

Testability is related to our framework through the following observation: 

Proposition 2.12 (a) A simple graph parameter f is testable if and only if f{Gn) converges for 
every convergent graph sequence {Gn) with \ V{Gn)\ oo. 

(b) A sequence (Gn) of simple graphs with \V{Gn)\ ^ oo is convergent if and only if f{Gn) 
converges for every testable simple graph parameter f . 

Using the notions and results concerning graph distance and convergence above, we can give 
several characterizations of testable parameters, see Section [61 

Property testing, mentioned in the introduction, is related to parameter testing in many 
ways. For example, Fischer and Newman [23] proved that the edit distance (see Section 14.31) 
from the set of graphs exhibiting a testable property "P is a testable parameter. See also [1] for 
a characterization of testable graph properties. 

3 Graphons 

In [27], Lovasz and Szegedy introduced graphons as limits of left- convergent graph sequences. 
Here we will first study the space of graphons in its own right, defining in particular a general- 
ization of the distance 5u to graphons, and state the analogue of Theorem 12.71 for graphons. The 
discussion of graphons as limit objects of left- convergent graph sequences will be postponed to 
the last subsection of this section. 

3.1 Homomorphism Densities 

Let W denote the space of all bounded measurable functions W : [0, 1]^ — M that are symmetric, 
i.e., W{x,y) = W{y,x) for all x,y e [0,1]. Let W[o,i] be the set of functions W e W with 
< W{x, y) < 1. More generally, for a bounded interval / C M, let W/ be the set of all functions 
W & W with W{x,y) G /. Given a function W G W, we can think of the interval [0, 1] as the 
set of nodes, and of the value W{x, y) as the weight of the edge xy. We call the functions in W 
graphons. 

We call a partition P of [0, 1] measurable if all the partition classes are (Lebesgue) measurable. 
The partition is an equipartition if all of its classes have the same Lebesgue measure. A step 
function is a function W E W for which there is a partition of [0, 1] into a finite number of 
measurable sets Vi, . . . ,Vk so that W is constant on every product set Vi x Vj. We call the sets 
Vi the steps of the step function. Often, but not always, we consider step functions whose steps 
are intervals; we call these interval step functions. If all steps of a step function have the same 
measure 1/k, we say that it has equal steps. 
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Every graphon W defines a simple graph parameter as follows [27]: If F is a simple graph 
with V{F) = {!,..., k}, then let 



t{F,W) = 




Every weighted graph G with nodes labeled 1, . . . ,n defines an interval step function Wq 
such that t{F,G) = PFg): We scale the nodeweights of G to sum to 1. Let Ji = [0,ai(G')], 
h = {(yi{G), ai{G) + 0:2(6')], . . . , and J„ = [ai{G) + ■ ■ ■ + an-i{G), 1]. We then set 

Wcix^y) = (3y(x)v{y){G) 

where v{x) = i whenever x G /j. (Informally, we consider the adjacency matrix of G, and replace 
each entry by a square of size ctj x aj with the constant function (3ij on this square.) If G 
is unweighted, then the corresponding interval step function is a 0-1-function with equal steps. 
With this definition, we clearly have Wg G W, and 

t{F,G)=t{F,WG) (3.2) 

for every finite graph F. The definition (13.11) therefore gives a natural generalization of the 
homomorphism densities defined in (12.21) and (12. 3p . 

Using this notation, we can state the main result of [27] : 

Theorem 3.1 For every left- convergent sequence (Gn) of simple graphs there is a graphon W 
with values in [0, 1] such that 

t{F,Gn)^t{F,W) 

for every simple graph F. Moreover, for every graphon W with values in [0, 1] there is a left- 
convergent sequence of graphs satisfying this relation. 



3.2 The Cut Norm for Graphons 



The distance da of two graphs introduced in Section [2731 was extended to graphons by Frieze and 
Kannan [5^ . It will be given in terms of a norm on the space W, the rectangle or cut norm 



\W\\ci = sup 

5,TC[0,1] 



W{x, y) dxdy 



SxT 



sup 

f,g: [0,1]^[0,1] 



W{x,y)f{x)g{y) dx dy 



(3.3) 



where the suprema go over all pairs of measurable subsets and functions, respectively. The cut 
norm is closely related to L^o Li norm of W, considered as an operator on -^^^([0, 1]): 



00^1= sup / W{x,y)f{x)g{y)dxdy. 

f,g: [0,1]^[-1,1] J[0,l]2 



(3.4) 
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Indeed, the two norms are equivalent: 

^||W^||oo^i< ||W^||n< llW^lloo^i. (3.5) 

See section 17.11 for more on connections between the cut norm and other norms. 

It is not hard to see that for any two weighted graphs G and G' on the same set of nodes and 
with the same nodeweights, 

du{G,G') = \\WG-Wa\\u. (3.6) 

where Wg denotes the step function introduced in Section 13. 1[ The cut-norm therefore extends 
the distance du from weighted graphs to graphons. 

We will also need the usual norms of as a function from [0, 1]^ M; we denote the 
corresponding Li and L2 norms (with respect to the Lebesgue measure) by and ||Vr||2- 

The norm ||.||2 defines a Hilbert space, with inner product 

{U,W)=[ U{x,y)W{y,x)dxdy. 
3.3 Approximation by Step Functions 

We need to extend two averaging operations from graphs to graphons. For W ^ W and every 
partition V = (Vi, . . . , Vq) of [0, 1] into measurable sets, we define a weighted graph on q nodes, 
denoted by W/V, and called the quotient of W and V, by setting ai(W/V) = A(l^) (where A 
denotes the Lebesgue measure) and 

In addition to the quotient W/V, we also consider the graphon Wp defined by 

Wp{x, y) = l3ij{W/V) whenever x G 1/i and y G Vj. (3.8) 

It is not hard to check that the averaging operation W ^— W-p is contractive with respect to 
the norms introduced above: 

IIW^pIId < llW^lln, ||W^p||i < llW^lli and ||Wp||2 < llW^b- (3.9) 

The graphon W-p is an approximation of by a step function with steps V. Indeed, it is the 
best such approximation, at least in the L2-norm: 

\\W -Wv\\2 = m:m\\W -Uv\\2 (3.10) 

U-p 

where the minimum runs over all step functions with steps V (this bound can easily be verified 
by varying the height of the steps in f/p). While it is not true that W-p is the best approximation 
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of W with step in V in the cut-norm, it is not off by more than a factor of two, as observed in 

\\W -Wp\\n<2mm\\W -Up\\a (3.11) 

U-p 

where the minimum runs over all step functions with steps V. Indeed, combining the triangle 
inequality with the second bound in fl3.9p and the fact that {Up)p = U-p, we conclude that 
\\W - Wp\\n < \\W - Up\\n + \\U-p - W-p\\n < \\W - U-p\\n + \\U-p - W\\n, as required. 

The definition of W-p raises the question on how well Wp approximates W. One answer to this 
question is provided by following lemma, which shows that W can be approximated arbitrarily 
well (pointwise almost everywhere) by interval step functions with equal steps. (The lemma is 
an immediate consequence of the almost everywhere differentiability of the integral function, see 
e.g. Theorem 7.10 of [30].) 

Lemma 3.2 For a positive integer n, let Vn he the partition of [0, 1] into consecutive intervals 
of length 1/n. For any W G W, we have Wp^ — > W almost everywhere. 

While the previous lemma gives the strong notion of almost everywhere convergence, it does 
not give any bounds on the rate of convergence. In this respect, the following lemma extending 
the weak Regularity Lemma from graphs to graphons, is much better. In particular, it gives a 
bound on the convergence rate which is independent of the graphon W . 

Lemma 3.3 ( |22] ) For every graphon W and every £ > 0, there exists a partition V of [0, 1] 
into measurable sets with at most 4^/^^ classes such that 

\\W -Wp\\u < e\\W\\2. 

With a slightly weaker bound for the number of classes, the lemma follows from Theorem 12 
of [22] and the bound (13.111) . or the results of [29]. As stated, it follows from Lemma [7.31 in Sec- 
tion [721 which generalizes both Lemma [X^ and the analogous statement for graphs. Lemma [531 
The Szemeredi Regularity Lemma [31] also extends to graphons in a straightforward way. See 
[2^ for this and further extensions. 

At the cost of increasing the bound on the number of classes. Lemmas 12.41 and 13.31 can be 
strengthened in several directions. In this paper, we need the following form, which immediately 
follows from Lemmas 12.41 and 13.31 by standard arguments, see appendix for the details. 

Corollary 3.4 Let e > 0, and q > 2^°/^^ Then the following holds: 

(i) For all graphons W , there is an equipartition V of [0, 1] into q measurable sets such that 

\\W -Wp\\u < s\\W\\2. 
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// we impose the additional constraint that V refines a given equipartition V of [0, 1] into k 
measurable sets, it is possible to achieve this bound provided q is an integer multiple of k and 
q/k> 220/-'. 

(ii) For all weighted graphs G on at least q nodes there exists a partition V = (Vi, . . . , Vg) of 
V{G) such that 

da{G,Gr)<s\\G\\2 

and 

au{G) < amax(G') for all i = 1, . . . ,q. (3.12) 

(1 



3.4 The Metric Space of Graphons 

We now generalize the definition of tlie cut-distance (12.71) from graphs to graphons. 

Let Ai denote the set of couphngs of the uniform distribution on [0, 1] with itself, i.e., the 
set of probability measures on [0, 1]^ for which both marginals are the Lebesgue measure. (This 
is the natural generalization of overlays from graphs to graphons). We then define: 

So{W, W) = inf sup / (W{x, y) - W'{u, v)) dfi{x, u) dfi{y, v] 
Atex 5 Tc[o,il2 J 



(y,v)eT 



For two step functions, finding the optimal "overlay" can be described by specifying what 
fraction of each step of one function goes onto each step of the other function, which amounts to 
fractional overlay of the corresponding graphs. Hence the distances of two unlabeled weighted 
graphs and the corresponding interval step functions are the same: 

dniG,G')=6aiWG,WG'). (3.13) 

It will be convenient to use the hybrid notation 5n(f/, G) = 5n(f/, Wg)- 

The next lemma gives an alternate representation of the distance 5n(VF, W), in which "over- 
lay" is interpreted in terms of measure-preserving maps rather than couplings. We need some 
definitions. Recall that a map : [0, 1] — [0, 1] is measure-preserving, if the pre-image (j)~^{X) is 
measurable for every measurable set X, and X{(j)~^{X)) = X{X). A measure-preserving bisection 
is a measure-preserving map whose inverse map exists and is also measurable (and then also 
measure-preserving). Finally, we consider certain very special measure-preserving maps defined 
as follows: Let us consider the partition Vn = iVi-, ■ ■ ■ , Vn) of [0, 1] into consecutive intervals of 
length 1/n, and let vr be a permutation of [n]. Let us map each Vi onto Vn(i) by translation, to 
obtain a piecewise linear measure-preserving map vf : [0, 1] [0, 1]. We call vr an n-step interval 
permutation. 

For M/ e W and : [0, 1] ^ [0, 1], we define W''' by W'''{x,y) = W {(f){x) , (piy)) . 
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Lemma 3.5 Let U,W eW. Then 

6a{U,W) = M\\U^ -VV^Wa (3.14) 

<t>,i> 

{where the infimum is over all measure-preserving maps (p^if) : [0, 1] [0, 1]) 

= inf ||f/- ly^'lln (3.15) 

{where the infimum is over all measure-preserving bijections ip : [0, 1] [0, 1]) 
= lim mm\\U -W^Wn (3.16) 

n— >oo 7r 

{where the minimum is over all permutations ir of [n].) 

The proof of the lemma is somewhat tedious, but straightforward, see appendix for details. 

Note that, for W G W, for an n-step interval permutation vf, and for the partition Vn of 
[0, 1] into consecutive intervals of lengths 1/n, the graph /Vn is obtained from W/Vn by a 
permutation of the nodes of W/Vn- As a consequence, the identity (I3.16P is equivalent to the 
following analogue of (12. 8p for graphons: 

HU, W) = lim 6a{U/Vn, W/Vn). (3.17) 



Using Lemma [3751 it is easy to verify that 6n satisfies the triangle inequality. Strictly speaking, 
the function So is just a pre-metric, and not a metric: Formula (13.151) implies that ^□(VF, W) = 
whenever W = for some measure preserving transformation </> : [0, 1] [0, 1]. Nevertheless, 
for the sake of linguistic simplicity, we will often refer to it as a distance or metric, taking the 
implicit identification of graphs or graphons with distance zero for granted. Note that the fact 
that there are graphons W, W G W which are different but have "distance" zero is not just a 
peculiarity of the limit: for simple graphs 6o{G,G') is zero if, e.g., G' is a blow-up of G (cf. 
(12. 9p ). We'll say more about graphons with distance in the next section. 

It was proved in [29] that the metric space (W[o,i],5n) is compact. Since W/ and W[o,i] are 
linear images of each other, this immediately implies the following proposition. 

Proposition 3.6 Let I be a finite interval, and let W/ be the set of graphons with values in I. 
After identifying graphons with 6\j distance zero, the metric space {Wi,6o) is compact. 

3.5 Left Versus Metric Convergence 

We are now ready to state the analogue of Theorems 12.61 and 12.71 for graphons. We start with 



the analogue of Theorem 12.71 

Theorem 3.7 Let W, W E W, let C = max{l, ||iy||oo, ||W^'||oo}, and letk>l. 
(a) If F is a simple graph with m edges, then 

\t{F, W) - t{F, W')\ < AmC"'-^6o{W, W). {3.U 
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(b) // \t{F, W) — t{F, W')\ < 3 ^ for every simple graph F on k nodes, then 

22(7 

6a{W,W')<—==. 

The first statement of tliis tlieorem is closely related to the "Counting Lemma" in the theory 
of Szemeredi partitions, and gives an extension of a similar result of [27] for functions in VV[o,i] to 
general graphons. It shows that for any simple graph F, the function W i— > t{F, W) is Lipschitz- 
continuous in the metric (5n, and is reasonably easy to prove. By contrast, the proof of the second 
one is more involved and relies on our results on sampling. 

In particular, we will need an analogue of Theorems 12.91 and 12. 101 to sampling from graphons. 
These theorems are stated and proved in Section 1^31 (Theorem 14.71 and 14.61) . Using Theorem 14. 7[ 
we then prove Theorem 13.71 in Section 14. 6[ 

Theorem 13.71 immediately implies the analogue of Theorem 12.61 for graphons: 

Theorem 3.8 Let I be a finite interval and let (Wn) be a sequence of graphons in W/. Then 
the following are equivalent: 

(a) t{F, Wn) converges for all finite simple graphs F; 

(b) Wn is a Cauchy sequence in the 5n metric; 

(c) there exists aW& W/ such that t{F, Wn) — > t{F, W) for all finite simple graphs F. 

Furthermore, t{F, Wn) — ^ t{F, W) for all finite simple graphs F for some W if and only 
^fSa{Wn,W)^0. 

Note that by equations (13. 2p and (I3.13p . Theorems 12.61 and 12.71 immediately follow from 
Theorems 13.81 and 13. 7[ Together with Theorem 13.11 these results imply that after identifying 
graphons with distance zero, the set of graphons VV[o,i] is the completion of the metric space of 
simple graphs. Proposition 13.61 equation (13.131) and Lemma 13.21 easily imply that the existence 
of the limit object (Theorem 13.11) can be extended to convergent sequences of weighted graphs: 

Corollary 3.9 For any convergent sequence (Gn) of weighted graphs with uniformly bounded 
edgeweights there exists a graphon W such that So(Wg„,W) 0. Conversely, any graphon W 
can be obtained as the limit of a sequence of weighted graphs with uniformly bounded edgeweights. 
The limit of a convergent graph sequence is essentially unique: If Gn W , then also Gn W' 
for precisely those graphons W' for which 5n(W, W') = 0. 

As another consequence of Theorem 13.81 we get a characterization of graphons of distance 0: 

Corollary 3.10 For two graphons W and W' we have (5n(VF, W') = if and only if t{F, W) = 
t{F, W') for every simple graph F. 
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Another characterization of such pairs is given in [19]: (5n(iy, 1^') = if and only if there 
exists a third graphon U such that W = If^ and W = for two measure-preserving functions 
0, ip : [0, 1] [0, 1] (in other words, the infimum in fl3.14l) is a minimum if the distance is 0). 

3.6 Examples 

Example 3.1 (Random graphs) Let G{n,p) be a random graph on n nodes with edge density 
< p < 1; then it is not hard to prove (using high concentration results) that the sequence 
(G(n,p), n = 1, 2, . . . ) is convergent with probability 1. In fact, t(F, G{n,p)) converges to 
with probabihty 1, and so (with probability 1) G{n,p) converges to the constant function W = p. 

Example 3.2 (Quasirandom graphs) A graph sequence is quasirandom with density p if and 
only if it converges to the constant function p. Quasirandom graph sequences have many other 
interesting characterizations in terms of edge densities of cuts, subgraphs, etc. [20]. In the second 
part of this paper we'll discuss how most of these characterizations extend to convergent graphs 
sequences. 

Example 3.3 (Half-graphs) Let ifn,n denote the bipartite graph on 2n nodes 
{1, . . . ,n,l' , . . . ,n'}, where i is connected to j' if and only if i < j. It is easy to see 
that this sequence is convergent, and its limit is the function 



Example 3.4 (Uniform attachment) Various sequences of growing graphs, motivated by 
(but different from) internet models, are also convergent. We define a (dense) uniform attach- 
ment graph sequence as follows: if we have a current graph (?„ with n nodes, then we create a 
new isolated node, and then for every pair of previously nonadjacent nodes, we connect them 
with probability 1/n. 

One can prove that with probability 1, the sequence (Gn) has a limit, which is the function 
W{x,y) = min{x,y). From this, it is easy to calculate that with probability 1, the edge density 
of Gn tends to f W = 1/3. More generally, the density of copies of any fixed graph F in G{n) 
tends (with probability 1) to t{F, W), which can be evaluated by a simple integration. 

4 Sampling 

4.1 Injective and Induced Homomorphisms 

In order to discuss sampling, we will consider not only the number of homomorphisms defined 
earlier, but also the number of injective and induced homomorphisms between two simple graphs 
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F and G. We use inj(F, G) to denote the number of injective homomorphisms from F to G, 
and ind(-F, G) to denote the number of those injective homomorphisms that also preserve non- 
adjacency (equivalently, the number of embeddings of F into G as an induced subgraph). 

We will need to generalize these notions to the case where G is a weighted graph with 
nodeweights one and edgeweights Pij{G) E M, where we define 

inj(F,G)= Yl n f3HuUiv)iG) (4.1) 

<^eInj(F,G) uv£E{F) 

and 

ind(F,G)= Yl n f^t>iuUiv)iG) n {^-PHuUiv)iG)). (4.2) 

^elni(F,G)uveE{F) uv&E{F) 

Here Inj(F, G) denotes the set of injective maps from V{F) to V{G), and E{F) consists of all 
pairs {u,v} of distinct nodes such that uv ^ E{F). We also introduce the densities 

^in,(i^,G) = -^|!;^^ and tUF,G) = j^^^^^ (4.3) 

{\^ {G)\)\V{F)\ {\^ [G)\)\V(F)\ 

where {n)k = n{n — 1) ■ ■ ■ {n — k + 1). 

The quantities t{F, G), tinj(-F, G) and tind(-^, G) are closely related. It is easy to see that 

tinj(F, G)=J2 tUF', G) and tUF, G) = ^ (-l)l''^^'^^''^^^ltinj(F', G) (4.4) 

F'DF F'DF 

whenever F is simple and G is a weighted graph with nodeweights ai(G) = 1. The quantity 
t{F, G) is not expressible as a function of the values tinj(-^, G) (or tind(-^, G)), but for large graphs, 
they are essentially the same. Indeed, bounding the number of non-injective homomorphisms 
from V{F) to V{G) by ('^f |\/(G) one easily proves that 



|t(F,G)-t..j(F,G)|<^("^f)")||G||r)l. 



(4.5) 



If all edgeweights of G lie in the interval [0, 1], this bound can be strengthened to 



1 f\V{F)\\ 



|t(F,G)-ti,j(F,G)|<^^(^' ^2 J' (4.6) 



see [27] for a proof. 



4.2 Sampling Concentration 

We will repeatedly use the following consequences of Azuma's Inequality: 



21 



Lemma 4.1 Let {Q,A, P) be a probability space, let k be a positive integer, and let C > 0. 

(i) Let Z = {Zi, . . . , Zk), where Zi, . . . , Zk are independent random variables, and Zi takes 
values in some measure space {Qi,Ai). Let / : Qi x ■ ■ ■ x Qj. ^ be a measurable function. 
Suppose that \f{x) — f{y)\ < C whenever x = (xi, . . . , Xk) and y = {yi, . . . ,yk) differ only in one 
coordinate. Then 

P(J{Z) > E[/(Z)] + Ac) < e-^'/2fc P(|/(^) - E[/(Z)]| > Ac) < 2e-^'/2fe^ (4 7) 

(a) The bounds (14.71) also hold if Zi, . . . , Z^ are chosen uniformly without replacement from 
some finite set V and \f{x) — f{y)\ < C for all x and y which either differ in at most one 
component, or can be obtained from each other by permuting two components. 

Proof. Let Mj = E[f{Z) \ Zi, . . . , Zj]. Then Mq, . . . , Mj. is a martingale with bounded 
martingale differences (for the case (ii) this requires a little calculation which we leave to the 
reader). The statement now follows from Azuma's inequality for bounded martingales. □ 



4.3 Randomizing Weighted Graphs 

Given a weighted graph H with nodeweights 1 and edgeweights in [0, 1], let G{H) denote the 
random simple graph with V{G) = V{H) obtained by joining nodes i and j with probability 
j3ij{H) (making an independent decision for every pair ij, and ignoring the loops in H). 

We need two simple properties of this well known construction. To state the first, we define 
the edit distance di of two weighted graphs with the same node set [n] and nodeweights 1 as 

1 " 

Lemma 4.2 Let Hi and H2 be two weighted graphs on the same set of nodes with nodeweights 
1 and with edgeweights in [0, 1]. Then G{Hi) and G{H2) can be coupled so that 

E{di{G{Hi),G{H,))) = diiHi,H2). 

Proof. For every edge ij, we couple the decisions about the edge so that in both graphs 
this edge is inserted with probability min(/5jj(ifi), /5ij(if2)) and missing with probability 1 — 
ma.x{j3ij{Hi), Pij{H2)). So the probability that the edge is present in exactly one of G{Hi) and 
G{H2) is \Pij{Hi) — Pij{H2)\, which proves the lemma. □ 

Lemma 4.3 Let H be a weighted graph on n nodes with nodeweights 1 and with edgeweights in 
[0,1]. Then 

Fr( da{H, G{H)) < ^ ) > 1 - 2"". 
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Proof. Let V{H) = V{G{H)) = V, let /i = and let H be the graph obtained from H 

by deleting all diagonal entries in (3{H). Fix two sets S*, T C For i ^ j eV , let 



1 iiij eE{G{H)), 
otherwise. 



Observe that the expectation of eG(H){S,T) is e^{S,T). Since eG{H){S,T) is a function of the 
independent random variables {Xij)i^j that changes by at most 2 if we change one of these 
variables, we may apply Lemma Wl] to conclude that 

Pr(|eo(^)(5,T) - 6^(5, T)| > ^,n^)) < 2exp(-^^|^) < exp(-^). 

(Here we used that e~^^"/^ < 1/2 in the last step). Taking into account that the number of pairs 
{S,T) is 4", we concluded that the probability that dniH ,G{H)) < /i = 3/y/n is larger than 
1 _ 4ng-M2„2/4 > I _ 2-n_ Since dn{H, H) < l/n < 1/v^, this completes the proof. □ 



4.4 VF-random Graphs 

Given a graphon W and a subset S C [0, 1], we define the weighted graph W[S] on node set 
S, all nodes with weight 1, in which /3a;y(iy[S']) = W{x,y). If G W[o,i], then we can construct 
a random simple graph W[S\ on S by connecting nodes Xj and Xj with probability W{Xi^Xj) 
(making an independent decision for every pair). 

This construction gives rise to two random graph models defined by the graphon W . For 
every integer n > 0, we generate a W-random weighted graph H(n, W) on nodes {1, . . . ,n} as 
follows: We generate n independent samples Xi, . . . ,X„ from the uniform distribution on [0, 1], 
and consider Vr[{Xi, . . . , X„}] (renaming i the node Xi). If G W[o,i], then we also define the 
W-random (simple) graph G(n, W) = W[{Xi, . . . , X„}]. 

When proving concentration, it will often be useful to generate G(n, W) by first independently 
choosing n random variables Xi, . . . , X„ and n{n + l)/2 random variables Yij {i < j) uniformly 
at random from [0, 1], and then defining G(n, W) to be the graph with an edge between i and 
j whenever Y^j < W{Xi, Xj). This allows us to express the adjacency matrix of G{n,W) as 
a function of the independent random variables Zi = (Xi,Yii), Z2 = (X2, Y12, F22), ■ . . Zn = 
{Xn, Yin, ^2n5 ■ ■ ■ , ^nn), as required for the application of Lemma 14.11 (i). 

It is easy to see that for every simple graph F with k nodes 

E(ti,j(F,G(n,W^))) = E{ti^i{F,H{n,W)))=t{F,W), (4.8) 

where the second equality holds for all W while the first requires W G W[o,i]. From this we get 
that 

\E{tiF,H{n,W)))-t{F,W)\<-rj if ||W-||oo < 1, (4.9) 
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and 



\E{t{F,Gin,W)))-t{F,W)\<^(^^ if WeWio,i]. (4.10) 

Concentration for the l^-random graph G{n, W) was estabhshed in Theorem 2.5 of [27]. To 
get concentration for l^-weighted random graphs, we use Lemma 14.11 This gives the following 
lemma, which also slightly improves the bound of Theorem 2.5 of [27] for VT-random graphs. See 
the appendix for details of the proof. 

Lemma 4.4 Let F be a simple graph on k nodes, let < e < 1 and let W eW. Then 

p{\t{F,U{n,W))-tiF,W)\ > e) < 2exp (-jf^^) ^/ IIW^IU < 1, (4.11) 



and 

p[\t{F,G{n,W))-tiF,W)\>e') <2exp(^-^n^ if W eWyo,i]. (4.12) 

From this lemma we immediately get the following: 

Theorem 4.5 (a) For any W G W, the graph sequence H(n, ly) is convergent with probability 
1, and its limit is the graphon W . 

(b) For any W G VV[o.i] , the graph sequence G(n, W) is convergent with probability 1, and its 
limit is the graphon W . 

4.5 The Distance of Samples 

The closeness of a sample to the original graph lies at the heart of many results in this and the 
companion paper [1^. We will prove these results starting with an extension of Theorem I2.10[ 
which is an improvement of a theorem of Alon, Fernandez de la Vega, Kannan and Karpinski 
[Hill], as discussed earlier. 

Theorem 4.6 Let k be a positive integer. 

(i) IfU eW, then 

||H(fc,[/)||n-||f/||n| <i^||f/|U (4.13) 

with probability at least 1 — 2e^^/®. 

(ii) IfUi,U2 € VV[o.i], then G{k,Ui) and G{k,U2) can be coupled in such a way that 

da{Gik,U,),Gik,U2))-\\Ui-U2\\a < (4.14) 



with probability at least 1 — e 
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Proof. 

that 



(i) Since the assertion is homogeneous in U, we may assume without loss of generahty 
„ = 1. The proof proceeds in two steps: first we prove that for k > 10^, we have 



H(fc,f/)||n) 



□ < 



(4.15) 



and then use Lemma 14.11 to prove concentration. 

It turns out that the most difficult step is the proof of an upper bound on the expectation of 
||H(/c, t^)||n. To obtain this bound, we use a refinement of the proof strategy of [3], using Lemma 
3 from that paper as our starting point. The main difference between their and our proofs is 
that we first bound the expectation of ||H(/i;, f/)||n, and only use Lemma [4.11 in the very end. 
This allows us to simplify their proof, giving at the same time a better dependence of our errors 
on k. 

Let Xi, . . . , Xfc be i.i.d. random variables, distributed uniformly in [0, 1], let B = -B(X) be the 
k X k matrix with entries Bij = U{Xi,Xj), and for 51,52 C [k], let B{Si,S2) = J2ieSi j(^S2 ^"i-i- 
Finally, given a set 5 C [k], let P{S) be the set of nodes i G [k] such that B{{i},S) > 0, and 
N{S) be the set of nodes for which B{{i}, S) < 0. We will prove upper and lower bounds on the 
expectation of ||i?||n = ma,Xsj^^s2C[k] \B{Si, S2)\. 

The lower bound is a simple sampling argument: consider two measurable subsets Ti,T2 C 
[0, 1], and let Si = {i & [k] : Xi G Ti}, and similarly for S2. Then 

k- 



E||H(fc,[/)||n>- E B{Si,S2) 



k 



U{x,y) dx dy + — / U{x,x)dx 
T1XT2 ^ JTi_r\T2 



> 



U{x,y) dx dy 



T1XT2 



2 
k' 



Taking the supremum over all measurable sets 6*1, 5*2 C [0, 1], this proves the lower bound in 

To prove an upper bound on the expectation of ||H(/i;, f/)||n, we start from Lemma 3 of [5]. 
In our context, it states that for a random subset Q C [k] of size p, we have 

BiSi, S2) < Eq[b{p{q n 52), S2)] + -^\\b\\f, 



where ||-B||f = x/J^ij-^ij — ^- Inserting this inequality into itself, we obtain that 

B{Si,S2) < Eq, 



max5(P(Ti),P(T2)) 



2e 



In order to take the expectation over the random variables Xi, . . . , X^, it will be convenient 
to decompose P{Ti) and P(T2) into the parts which intersect Q = QiU Q2 and the parts which 
intersect = [k] \ {Q1UQ2). Let Pq{T) = P(T) HQ and Pq^T) = P{T)\Q. Since \P{T)\ < k 
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and \P{T)\Pqc{T)\ = \Pq{T)\ < 2pwe have that B{P{T^),P{T2)) < B{Pqc{T,), PQc{T2))+Apk, 
implying that 



5(^1,^2) < Eq„q, max5(PQc(Ti),PQ.(T2)) 



2e 
Vp 



+ Apk. 



Applying this estimate to —B and taking the maximum of the two bounds, this gives 



B\\n<^c 



,Q2 



1 2P 

maxmax{5(PQc(ri),PQc(T2)),-5(iVQc(Ti),iVQc(T2))} + — + Apk, (4.16) 



where Nqc{T) = N{T)\Q. 

Consider a fixed pair of subsets Qi,Q2 C [k]. Fixing, for the moment, the random variables 
Xq, let us consider the expectation of, say, B{Pqc(Ti), Pqc(T2)). For T C [k], let Xt be the 
collection of random variables X with i E T, and let 



P(Xt)= {a:G[0,l]: J]f/(x,Xi)>0}. 



Then PQc(r,) = {j e Q': X, E P(XtJ}, and 



B{Pq.{T,),Pq4T2)) =|Q^|(|g'=|-l) / U{x,y)dxdy 

7'(XtJxP(Xt2) 



+ IQ'^l / f/(x,x) rfx 

ip{XTi)nP{XT2) 

< A;lf/||n + A;. 

It is not hard to see that the random variable Y = B{Pqc{Ti), Pqc{T2)) is highly concentrated. 
Indeed, consider F as a function of the random variables in Xqc If we change one of these 
variables, then Y changes by at most Ak, implying that with probability at least 1 — e'^p^l'^'^ ^ 
Y < E{Y) +pk'^. The same bound holds for the random variable Y = —B{Nqc(Ti), Nqc[T2)). As 
a consequence, the maximum in (14.161) obeys this bound with probability at least 1— 2^P+^e"'^'^^/'^^. 
Since Y,Y < k'^ for all X, we conclude that 



\H{k,U)\\a 



IPI 



□ 



<\\U\\a + ^ + ^ + T + P + 2'^+'e~'p'''\ 
^/p k k 



Choosing p and p of the form p = [aVk] and p = (3k with a = (4v^log 2) and (3 
4(log2)^/'' + 4/10, this implies that for k > 10^, we have 



\H{k,U)\\a 



< 



□ 



^ ^8(log 2)^4 + 0.534...)) < \\U\\a+ ^ 



(4.17) 



which is the upper bound in (14.151) . 
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To prove concentration, we use that ||H(/i;, U)\\n changes by at most |||?7||oo if we change one 
of the random variables Xi, . . . , X^, so applying Lemma W7l\ (i) we get that its values are highly 
concentrated around its expectation: 



\H{k,U)\\a-E mk,U)\\a 



This completes the proof of (i). 

(ii) We couple G{k,Ui) and G{k, U2) as follows: as in the proof of (i) we chose Xi, . . . , Xk to 
be i.i.d., distributed uniformly in [0, 1]. In addition, we independently choose k{k + l)/2 random 
variables Yij = Yji uniformly at random from [0, 1]. In terms of these random variables, we then 
define Gi to be the graph on [k] which has an edge between i and j whenever Ui{Xi, Xj) < Yij, 
and G2 to be the graph which has an edge between i and j whenever U2{Xi, Xj) < Yij. Then 
Gi, G2 is a coupling of G{k, Ui) and G{k, U2), and 

dn{Gi,G2) = ^ max \B{Si,S2)\, 
k^ Si,S2C[k] 

where B is the matrix with entries Bij = lY,j<Ui{Xi,Xj) — '^Yij<U2{Xi,Xj)- We again have to bound 
the expectation of B{Pqc(Ti), Pqc(T2)). As before, we fix the variables Xi for i E Q, but now we 
also fix all random variables Yij for which {i,j} intersects Q. In order to calculate expectations 
with respect to the remaining random variables, we express B{Pqc(Ti), Pqc(T2)) as the sum 
^iJ&Q^ KjiTi,T2), where 

Bt,j{Ti,T2) = Bijli(zp(^Ti)'^j€P{Ti)- 

For i G Q'^, the event that i G P{T) for some T G Q can then be re-expressed as the event that 
Xi lies in the set 

jx G [0, 1] : 1y,^,<Ui(x,x^,) > Yl ^K,'<U2(x,x^,)y 
j'dT j'eT 

Observing that this set only depends on the fixed random variables, we can now proceed as before 
to calculate the expectation of i?,^^(Ti, T2), and hence the expectation of B{Pqc(Ti), Pqc(T2)). 
This, together with a similar (again much easier) calculation for the lower bound, leads to the 
estimate 

- ^ < E[d{G^,G2h] - \\Ui - U2\\n < (4.18) 

as before valid for k > 10^. Concentration is again proved with the help of Lemma [4. II □ 

Essentially the same proof also gives Theorem I2.10t 

Proof of Theorem 12.101 We generate the set S by choosing Xi, . . . ,Xk uniformly without 
replacement from V. In this way, we rewrite (in(G'i[5'], 6*2 [S]) in terms of the matrix B = B(X.) 
with entries Bij = (3xiXj{Gi) - (3xiXj{G2)- Observing that 

\Bij\ < G = 2max{||Gi||oo, 11^21100}, 
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we may proceed exactly as in the proof of Theorem 14.61 (i), leading to the bound 



Using Lemma [4.11 (ii) to prove concentration, we get the bound f l2.13p . 
We now come to the main theorem about sampling. 

Theorem 4.7 Let k he a positive integer. 

(i) IfUe W, then with probability at least 1 — e~''^^^'^^°^^''\ we have 

10 



(4.19) 
□ 



fc(f/,H(fc,f/))< 

(ii) IfUe >V[o,i], then with probability at least 1 - e~^'^/^'^^°^-^^\ we have 



6n{U,G{k,U)) < 



10 



Vlog2 k 



Proof. We again first bound expectations and then prove concentration, and as before, we 
assume without loss of generality that ||t/||oo < 1- Finally, we may assume that k > 2^^ > 10^, 
since otherwise the bounds of the theorem are trivial. 

In a first step, we use the weak Szemeredi approximation in Lemma 13.31 and the sampling 
bound fl4.15p to show that it is enough to consider the case where ?7 is a step function. Indeed, 
given £ > 0, let [/i be a step function with q < 4^/^^ steps such that 



\U-UAa<e. 



(4.20) 



By the bound fl4.15p . we have that 



Soiii{k,U,),U{k,U)) < E[||H(A;,t/-f/i)||n] <£ + 
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As a consequence, 
E 



6a{U, H(fc, U)) < 6a{U, Ui) + E 6a{Ui, n{k, Ui)) + E Sa{H{k, U^), H(fc, U)) 

16 



< 2e 



(5n(f/i,H(A;,f/i)) 



(4.21) 



We are thus left with the problem of sampling from the step function Ui. Let Ui have steps 
Ji, ■ ■ ■ , Jq ^ [0, 1], and A( Jj) = a^. Let Xi, . . . , be independent random variables that are 
uniformly distributed on [0, 1], and let Zi be the number of points Xj that fall into the set Jj. It 
is easy to compute that 



E{Zi) = aik, Var(Zi) = (a^ - a- )A; < aik. 
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Construct a partition of [0, 1] into measurable sets J[, . . . , J'^ such that A(Jj') = Zi/k and 

X{Ji n J-) = min(ai, Zi/k), 

and also construct a symmetric function f/' G W such that the value of U' on J- x is the same 
as the value of Ui on Ji x Jj. Then [/' is a step function representation of H(/c, Ui), and it agrees 
with Ui on the set Q = Ulj^^{Ji n J/) x (Jj n Jj). Thus 



5i 



□ (f/i, H(A;, f/i)) < ||f/i - U'Wo < ||f/i - f/'lli < 2(1 - A(Q)) = 2(1 - (5^min(a„ |^))') 



< 4(1 - Emin(a. f )) = 2 f | < 2 E («- f )0 



1/2 



which we rewrite as 



(<5n([/i,H(fc,f/i)) <4gE(«* ^ 

j 

The expectation of the right hand side is 

i 

SO by Cauchy-Schwarz 



fc(f/i,H(A:,f/i)) 



' k ■ 



(4.22) 



Inserted into (14.211) . this gives 



6a{U,H{k,U)) 



16 An 
< 2f A I- W < 2f 



16 2 

+ 



21/.^ 



Choosing e = 2/(log2 k) and recalling that k > 2^^, this gives the upper bound 

Vlog2 A;\ . 6 



V^log^ 



(4.23) 



Observing that 5n(f/, H(A;, f/)) changes by at most 4/A; if one of the random variables Xj changes 
its value, we finally use Lemma [4.11 (i) to complete the proof of the first statement, 
(ii) The proof of the second statement is completely analogous: we first show that 



logs A; 



and then use Lemma 14.11 (i) to prove concentration. 



Vlog2 k ' 



(4.24) 



□ 



The proof again generalizes to samples from weighted graphs, this time leading to Theo- 
rem [2l9l 
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Proof of Theorem 12.91 Again, the proof is analogous to the proof of statement (i) above, 
except that we now use the original Frieze-Kannan Lemma (Lemma I2.4p for graphs instead of 
our Weak Szemeredi Lemma (Lemma 13.31) for graphons. One also needs to generalize the bound 
(I4.22P to samples Xi, . . . , Xk chosen uniformly without replacement from V{G), but this is again 
straightforward: the random variable Zj is now the number of points that fall into the class 
Vi of a. weak Szemeredi partition V = {Vi, . . . , Vq), and its variance is bounded by aik, where 
= Continuing as in the above proof, these considerations now lead to the bound 

SniG, G(fc, G))] < U + (16 + 2)^^^) < ^=1=, (4.25) 

J Vlog2 k V J ^log2 k 

where we again assume that the weights have been rescaled so that UGHoo = 1- Using Lemma WTLl 
(ii) to show concentration, this gives the bound (12.121) . □ 

4.6 Proof of Theorem [321 
4.6.1 Proof of Theorem [321 (a) 

Let V{F) = [n] and E{F) = {ei, . . . , Cm}- For t = 1, . . . , m, let it, jt be the endpoints of the 
edge et, and let Et = {ei, . . . , Ct} C E{F). Then t{F, W) — t{F, W) can be rewritten as 

t{F,W)-t{F,W')= I ( Yl W{xi,Xj)- Yl W'{xi,Xj)')dxi...dxn 

m „ 

= J2 ]JW{xi^,Xjj]JW\xi^,Xjj(w{xi^,XjJ -W'{xi,,Xj,)^ dxi...dxn. (4.26) 

Take any term in this sum, and for notational convenience, assume that it = 1 and jt = 2. 
Let X{xi, X3, . . . , Xn) be the product of those factors in n<i<t ^i^isy^js) Y[s>t ^'i^is^ ^js) that 
contain Xi, and let Y{x2, ■ ■ ■ , Xn) be the product of the rest. Then we have 

/ JJ Vr(a;i,,a;jJ JJ W( 

•^isi -^js) ( ^i-^iti -^jt) ^ i-^if) -^jt) I dXi . . . dXn 

jij .,„,,...,-,,,...»....)...... 

[0,l]"-2 [o,l]2 

Here the interior integral is bounded by 

j X(xi, . . . , Xn)Y{x2, ...,Xn) {W{Xi, X2) - W {Xi, X2)) dXi dX2 
[0,1]^ 

< \\X\U\Y\U\W-W'\\^^^. 



30 



Substituting into (14.261) . and using that 



\\X\U\Y\\^<\\W\\'^'\\W'\\Z^'<C"''-' 

and (by fl33|) ) 

\\W-W'\\^^,<A\\W-W'\\a 

we get 

\t{F,W) - t{F,W')\ < AmC^-^WW -W'Wa, 

Using the representation in Lemma [3.51 for the 60 distance and the fact that t{F, W) = t{F, W^) 
whenever is a measure-preserving function from [0, 1] to [0, 1], this bound imphes the bound 

dsn]). □ 

Remark 4.8 The above proof can easily be generahzed to show that 

|t(F,iy)|<4||l^||^(^)l-^||l^||n. (4.27) 

for all G W and all simple graphs F. Also, it is not hard to show that the factor 4 in (13.181) 
and (I4.27P is not needed if W and W are non-negative. 

4.6.2 Proof of Theorem [33 (b) 

Without loss of generality, we may assume that ||W^||oo7 ||W^'||oo < 1 (otherwise we just rescale 
both W and W by the maximum of these two numbers). Let U,U' G W[o,i] be the graphons 
f/ = liy + 1 and [/' = + |. Then 6d{W, W) = 2fa(f/, f/'), so it is enough to prove that 
6n{U,U') < n/^/hgJi. We will prove this bound by relating the distance of U and U' to the 
distance of the random graphs G{k, U) and G(/c, U'). To this end, we need an expression for the 
probability that G(/c, U) is equal to some given graph F on k nodes. We first use the relations 
(14. 8 p to express t{F, U) as a sum over graphs G on k nodes. Combined with (14. 4p and the fact 
that for all graphs F' and G on k nodes, tind(-^'7G') = unless G is isomorphic to F', we have 
that 

t(F, U) = 5^ p(g(A;, U)) = gY^^{F, G) 

G 



G F'dF 

= ^ E P{Gik,U))=F') (ind(F',F') 

■ F'DF 

where ind(-, ■) is defined in (14. 2p . With the help of inclusion-exclusion, this leads to 
p(G(fc,?7)) =f) =k\ 5^(-l)l^(^')\^(^)l(ind(F',F'))~V^',f/), 



F'dF 
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which in turn imphes that 

I P (g(^, U)) = f)-P {G{k, U')) = F)\<k\ Yl K^'' - ^(^'' 

F F,F': 

E{F')DE{F) 

where the sum runs over graphs F and F' on k nodes. To continue, we need to relate the 
homomorphism densities of U and U' to those of W and W. To this end, we insert the relation 
U = ^ (W + l) into the definition of t{F', U). For a graph F' on k nodes, this leads to the identity 

t{F\ U) = 2-l^(^')l Y ^(^"' ^) 

where the sum goes over all subgraphs that have the same node set as F' . Using the assumption 
of the theorem, we thus obtain the bound \t{F' , U) —t{F', U')\ < 3^'^^ which in turn implies that 

^|p(G(A;,f/)) = f) -P(^G(A;,f/')) =f)| < A;! Y 3"^' = fclS^^-'s^^^-i^/^^ 

F F,F': 

E{F)CE{F') 

Bounding k\ (rather crudely) by S'^^/^, we note that the right hand side is smaller than 3~''^'^. 

As a consequence, G{k, U) and G(/c, U') can be coupled in such a way that G(/c, U) = G{k, U') 
with probability at least 1 — 3"'^/^. Combined with the triangle inequality and the bound fl4.24p . 
we obtain that 

Sa{U, U') < E5u{G{k, U), G{k, U')) + £[5n{G{k, U), U)] + E[6a{G{k, U'), U')] 

10 11 



< 



Vlog2 k Vlog2 k 



□ 



5 Convergence in Norm and Uniform Szemeredi Parti- 
tions 

5.1 Comparison of Fractional and Non-Fractional Overlays 

Let G and G' be two graphs on n nodes, both with nodeweights 1/n. Consider any labeling that 
attains the minimum in the definition fl2.6p of (5n, and identify the nodes of G and G' with the 
same label. In this case, we say that G and G' are optimally overlaid. 

In addition to this distance we also defined the distance (5n, given in terms of fractional 
overlays X G X{G,G'), see (12.71) . Since every bijection between the nodes of Gi and G2 defines 
a fractional overlay X, we trivially have that 

6aiG,G')<6aiG,G'). 
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This inequality can be strict. Let G = K2, and let G' be a graph with two nonadjacent nodes but 
with a loop at each node. It is easy to see that 6ci{G,G') = 1/4, but 6ci{G,G') = 1/8 (the best 
fractional overlay is Xiu = 1/4 for all i G V{G) and u G V{G')). To give an example without 
loops, let G = -ft'3,3, and let G' consist of two disjoint triangles Ai and A2. There are only two 
essentially different ways to overlay these graphs; the better one maps two nodes of Aj into one 
color class of K^ ^ and the third one into the other color class. The number of edges in G' between 
Ai and A2 is 5, whence 6\j{G,G') > 5/36. (One can check that equality holds.) On the other 
hand, let us double each node in both graphs, to get G{2) = Kq q and G'{2) = Ai(2) U A2(2). 
Let us map one copy of each twin node of G'{2) into one color class of Kq q, and its pair into the 
other color class. Case distinction shows that the worst choice for S and T in the definition of 
dais S = V{Ai{2)) and T = y(A2(2)), showing that 

<5n(G,G0 < fa(G(2),G'(2)) = ^ = ^ < 

We have no example disproving the possibility that ^□(G'i,G'2) = 0(5n(Gi, 6*2)), but we are 
only able to prove a much weaker inequality given in Theorem 12.31 We start with a simple but 
very weak bound we will need. 

Lemma 5.1 Let Gi and G2 be weighted graphs on n nodes. If both Gi and G2 have nodeweights 
one, then 

fc(Gi,G2) <n%(Gi,G2). 

Proof. Let (X„j) be an optimal fractional overlay of Gi and G2, normalized in such a way that 
Y2^Xui = "^y^vj = ^/n for all n. We claim that there is a bijection vr : V{Gi) ^^(^2) such 
that Xutt{u) > for all u G V{Gi). This follows from a routine application of the Marriage 

Theorem: if there is no such bijection, then there are two sets S C V{Gi) and T C V{G2) such 
that 15*1 + |T| > n and Xgt < for all s G S* and all t But then 



^ - 5Z XI ^™ ~ 1^ XI + 1^ X 

n-^ n n n 

a contradiction. Thus a map vr with the desired properties exists. 
Let G'l be the image of Gi under this map. Then 

6a{Gi,G2) < max max \(3st{Gi) - (3^(^s)7r(t){G2)\ 

< max max n^Xs^(s)Xt^(t)\Pst{Gi) - p^^s)7rit){G2)\ 

s€V{Gi)t(^V{G2) w w\ 



<n^fa(Gi,G2), 
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which proves the lemma. □ 

Now we are able to prove Theorem 12.3^ which shows that the two distances 6 and 6 define the 
same topology. As the reader may easily verify, the proof below gives an exponent of 1/3 instead 
of the exponent of 1/67 from Theorem 12.31 if the number of nodes is large enough. However, even 
under this assumption, we could not obtain a linear bound, i.e., bi-Lipschitz equivalence. 

Proof of Theorem 12.31 The first inequality, as remarked before, is trivial. To prove the second, 
write SoiGi,G2y^^'^ = e. If n < e~^^ , then the bound follows by Lemma [5.11 and if e > 2/36, 
the bound is trivial, so we may assume that 

n > e~^^ and e < —. (5.1) 

16 

Consider an optimal overlay of and (in other words, consider a measure-preserving 
bijection </> : [0, 1] [0, 1]) such that 

\\WG,-Wij\a = 5o{G,,G2)=e<'\ 

Let us select a set Z of = [n/5] random points from [0, 1]. Let Hi = Wgi [Z] and H2 = Wq4>[Z]. 
Then by Lemma [4. 6[ we get that with probability at least 1 — e~sV^ > 1 — 8e^, 

Each element z & Z corresponds to a node iz G V{Gi) and a node jz G V{G2)- These pairs 
{izijz) form a bipartite graph F with color classes V{Gi) and V{G2) (which we assume are 
disjoint). 

Claim 5.2 With probability at least 1 — (2/e)~^", the bipartite graph F has a matching of size 
at least (1 — 2e)n. 

To prove this claim, we use Konig's Theorem: if F does not contain a matching of size 
(1 — 2e)n, then its edges can be covered by a set X of nodes with \X\ < (1 — 2e)n. Let 
Yi = V{Gi) \ X. Then there is no edge of F between Yi and 1^2- 

On the other hand, |li| + |y2| > (l + 2£:)n. Let Jj C [0, 1] be the union of intervals representing 
Yi in Wg,, so that A( Ji) + A( J2) > l+2e, and hence also A( Ji) + A(</)( J2)) >l + 2e. This implies 
that A(Ji n 0(^2)) > 25. The random set Z avoided this intersection; the probability of this 
happening is at most 

(1 - 2e)'' < e"2"^ < 

Since there are at most 4" pairs of sets (Fi, Y2), the probability that F does not have a matching 
of cardinality at least (1 — 2e)n is less than 4"'e~^"'. This proves the claim. 
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Now let ji) . . . , {imiirn) be a maximum matching in F . With positive probabihty, we have 
both dniHi, H2) < s^"^ + 20//c^/^ and m > (1 — Fix a choice of Z for which this happens. 

Let jm+i), • • • 5 {in,jn) bc an arbitrary pairing of the remaining nodes of V{Gi) and V{G2). 

We claim that the pairing ir : ir ^ jr gives an overlay of Gi and 6*2 with small d^i distance. 

Let S,TC ViG^), and let S' = S n {j,, . . . , T' = T n {ji, . . . , j^}. Then \S \ S'\ < ^, 
and |T \ T'| < and hence 

|eG;(^,T)-eG,(^,T)| < |e,;, (5',T') - eG,(^',TO| + 2(2^— + (— ) ). 

Here 

on 

and hence 

\eG'{S,T)-eG2{S,T)\ 8n 8n^ e^'k' 20kl 

- — ^ < 1 \ 1 

k k"^ 

<8e + 8e^ + e''{l + e'^ + 20(1 + e-^/^n-'/' 
<8e + 8e^ + e^^l + ef + 20^(1 + eY^^ < 32e 

where we used ( 15. ip in the last two bounds. □ 
5.2 Convergence in Norm 

Let {Gn) be a convergent sequence of weighted graphs. Theorem 13.81 then implies that there 
exists a graphon G W such that Wg„ —>■ W in the (5n distance. This does not imply, however, 
that Wcr, — »• in the || ■ ||n norm. It turns out, however, that the graphs in the sequence 
can be relabeled in such a way that this becomes true, provided has no dominant nodes in 
the sense that 

as n — > 00. This is the content of the following lemma, which will be useful when discussing 
testability. 

Lemma 5.3 Let {Gn) be a sequence of weighted graphs with uniformly bounded edgeweights, and 
no dominant nodes. If 

for some U G W, then the graphs in the sequence {Gn) can be relabeled in such a way that the 
resulting sequence (G^) of labeled graphs converges to U in the cut-norm: 

\\U -Wg' Ih^O. 



35 



Proof. We first prove the lemma for graphs with nodeweights one. Let m(n) = |V^(G„)|, and 
let Vm{n) be a partition of [0,1] into consecutive intervals of length l/m{n). By Lemma |3.2[ 
we have that \\U — f/-p„(„J|n 0, so combined with the assumption that S\j{U,Wg„) ^ we 
conclude that 5n(?7-p„(„), Wg„) — > 0. But the left hand side can be expressed as the distance of 
two weighted graphs on the same number of nodes, S\j{Up^^^^,Wg„) = S\j{U/Vm{n),Gn), so by 
Theorem I2.3[ we get that 

Sn{U/Vmin),Gn) ^0. 

This means that the graphs in the sequence (Gn) can be relabeled in such a way that for the 
resulting graph sequence, (G^), we have 

Combined with the fact that \\U — U-p^^^^^^Wo 0, this gives the statement of the lemma for 
graphs with nodeweights one. 

To prove the lemma for general sequences without dominant nodes, we use the Weak Regular- 
ity Lemma to approximate (G„) by a sequence of graphs (G„) with nodeweights one. Indeed, let 
us assume without loss of generality that all graphs in the sequence have total nodeweight 
= 1. Define a„ = maxj and choose in such a way that and a„2^°/'^" 

as n ^ oo. With the help of Corollary 13.41 (ii), we then construct a partition Vn of V{Gn) into 
Qn < 2^°/*^" classes such that (in(G„, (G„,)p„) — ^ and the classes in V„ have almost equal weights 
(in the sense that | ^^gy. ax{Gn) — J2yeVj ^yiGn)\ < o,n for all i,j G [qn])- Consider the sequence 
of graphs Gn that are obtained from Gn/Vn by changing all nodeweights to 1. Since the classes 
of Vn have almost equal weights, we have that \\Wq^ — WG„/v„\\a < Qn^^n which in turn 
implies that (5n(G„, Gn) 0. Thus (Gn) is a sequence of weighted graphs with nodeweights one 
which converges to U, implying that it can be reordered in such a way that — U\\ci ^ 0. 

But this means that Gn/Vn can be relabeled in such a way that HVTc^/t^^ — f/||n — > 0, which in 
turn implies that G„ itself can be relabeled so that HVFg^ — f^||n — ^ 0, as desired. □ 

The previous lemma suggests that we extend the definition of the distance 6 to the case when 
one of the arguments is a graphon U : 

6dU,G) = mm\\U -Wg'Wo (5.2) 

where the minimum goes over all relabelings G' of G. Then the lemma asserts that if Gn is a 
convergent graph sequence with uniformly bounded edgeweights and no dominant nodes, then 

HU,Gn)^0. 

In the special case of nodeweights one. Theorem 12.31 and Lemma 15.31 naively suggest the 
stronger statement that (5n(f/, G) can be bounded by a function /(5n(f/, Wg), \ V{G)\) such that 
f{x, n) — i> if X — s> and n oo. However, this is false, as the following example shows. 
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Example 5.1 Let G = Kn,n be the complete bipartite graph on 2n nodes, and let H = Knm,nm 
be the complete bipartite graph on 2nm nodes, where H has randomly labeled nodes, and let 
U = Wh- Then 6\z\{U, Wq) = 0. But it is not hard to show that for every n, if m is sufficiently 
large, then with large probability, \\Wh — WcWa > 1/10 for every relabeling G' of G, implying 
that Sci{U,G) > 1/10, see appendix for details. 

5.3 Convergent Szemeredi Partitions 

Given Lemma [573| we now are ready to prove Theorem 12.81 about the convergence of the quotient 
graphs of suitably chosen Szemeredi partitions for a convergent graph sequence (G„). 

We start by proving the easier direction, namely that convergent quotients imply convergence 
of the sequence (G„). 

Let e > 0, and let q be such that the conditions (i) and (ii) of Theorem 12.81 hold. For a 
fixed q, convergence of Gn/Vn is equivalent to convergence of all edgeweights and nodeweights, 
which in turn implies convergence in the 5u distance. Let no be such that duiGrjVn, Gm/Vm) < s 
whenever n,m> Uq, and > q whenever n > Uq. Then ^□(G'n, Gm) < 36 for all n,m > Uq 

by the triangle inequality, the property (i) and the fact that 5u{Gn, Gn/Vn) < rfn(G'„,, (G'„)p„). 
This proves that (G^) is a Cauchy sequence in the metric 5ui and hence left-convergent. 

To prove the necessity of the conditions (i) and (ii), consider a convergent sequence (Gn), 
a graphon U' such that f/') 0, and a constant e > 0. With the help of the Weak 

Regularity Lemma for graphons. Lemma [3.31 we can find a partition V of [0, 1] into go < 2^°/^' 
classes such that ||f/' — f/-p/||n < e/\/h. Applying a measure-preserving map to both U' and 
V , this allows us to find a graphon U and a partition V" of [0, 1] into go consecutive intervals 
such that \\U — t/-p"||n < s/\/^ and 5n(G„, U) — > 0. Appealing to Lemma [5.31 we finally relabel 
the graphs in the sequence (G„,) in such a way that for the relabeled sequence (which we again 
denote by (Gn,)), we get convergence in norm, 

as n — s> oo. 

On the other hand, by Lemma [2.41 we can find a sequence of partitions {V^ such that V'^ is 
a weakly (£:/v^)-regular partition of G.„ with not more than 2^^/^^ classes. Let g„ be the number 
of classes in P^, and let q = max„>ogn- By the bound (13. lip , we can refine the partitions V'n to 
obtain weakly {2e/V5 < £:)-regular partitions V'n with exactly q classes whenever |V^(G„)| > q. 
In a similar way, we can refine the partition V" to obtain a partition V of [0, 1] into q consecutive 
intervals such that 

\\U-U,h<±e<le. 
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Let no be such that for n > uq 

and let be the Lebesgue measure of the i^^ partition class of V. For n < no, we then set 
Vn = V^, and for n > riQ we define Vn to be the partition into the classes = {!,..., k^^}, 
= {A;i") + 1, . . . , fcj"^ + fc^"^}, etc., where the inte gers kl are chosen in such a way that 
[ai\V{Gn)\\ < fcf ^ < \ai\V{Gn)\]. With this definition, we have that 

\\{WgJv - l^(G„Kl|n < \\{Wg„)v - P^(g„kI|i < ^^7^ < ^ 

for all n > uq. Combined with the triangle inequality and the bound fl3.9p . this gives 

ddGn,{Gn)vJ = \\WG„-W^G.)rJb 

< \\Wg„ - U\\a + \\U - UrWn + \\Uv - {IYgMIu + \\{WgJv - W^G„)r„ !!□ 

< 2\\Wg„ -U\\a+\\U-Uv\\a+ \\{WgJv - W^g.)^^ \\^ < 2^ + + ^ = e 

whenever n > no- Thus Vn is a weakly e-regular partition of Gn, whether n < no, or n > riQ. By 
definition, it also is a partition into exactly q classes whenever > q. This proves (i). 

To prove (ii), we note that for n > no, we have 

daiGn/Vn, U/V) < - {WgM\u + KWg^v - UAu < ^^7^ + - U\\u. 

As n ^ 0, the right hand side goes to 0, as required. 



6 Parameter Testing 

6.1 Definitions and Statements of Results 

In this section we discuss the notion of continuous graph parameters, and the closely related 
notion of parameter testing. In parameter testing, one wants to determine some parameter of a 
large, simple graph G, e.g., the edge density or the density of the maximum cut. It is usually 
difficult to determine the exact value of such a parameter, but using sufficiently large samples, 
one might hope to approximate the value of parameter with large probability at a much lower 
computation cost; recall Definition (12. lip . 

Throughout this section, G will be a simple graph; this will be made explicit only in the state- 
ments of theorems and supplements. Our principal theorem gives several useful characterizations 
of testable graph parameters. 
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Theorem 6.1 Let f be a bounded simple graph parameter. Then the following are equivalent: 

(a) / is testable. 

(b) For every e > 0, there is an integer k such that for every simple graph G on at least k 
nodes, 

\f{G)-EU-{G{k,G))\<e. 

(c) For every convergent sequence {Gn) of simple graphs with \V{Gn)\ — ^ oo, the limit of 
f{Gn) exists as n ^ oo. 

(d) There exists a functional f{W) on W that is continuous in the rectangle norm, and 
f{WG)-f{G)^Ozf\V{G)\^oo. 

(e) For every e > there is an Eq > and an no G Z+ such that if Gi,G2 are two simple 
graphs with \V{Gi)\, |V^(G'2)| > '^o and ^□(G'l, G2) < £0, then \ f{Gi) - /(G2)| < £■ 

If we want to use (e) to prove that a certain invariant is testable, then the comphcated 
definition of the 6\j distance may cause a difficulty. So it is useful to show that (e) can be 
replaced by a weaker condition, which consists of three special cases of (e). For this purpose, 
we define the disjoint union G U G' of two graphs G and G' as the graph whose node set is the 
disjoint union of V{G) and V{G'), and whose edge set is E{G) U E{G'). 

Supplement 6.2 The following three conditions together are also equivalent to testability: 

(e.l) For every e > there is an e' > such that if G and G' are two simple graphs on the 
same node set and du{G,G') < e' , then \ f{G) — f{G')\ < e. 

(e.2) For every simple graph G, f{G[ni\) has a limit as m ^ 00. 

(e.3) /(GUA'i) - f{G) ^ if \V{G)\ ^ 00. 

We formulate two further conditions for testability, in terms of Szemeredi partitions. Recall 
that a partition {Vi, . . . , V^} of a finite set V is an equitable partition if [|l^|//cj < \Vi\ < \\V\/k~\ 
for every \ < i < k. Let V = {Vi, . . . , Vk} be an equitable partition of the node set of a simple 
graph G. A pair [Vi, Vj) if partition classes is called an e-regular pair, if for all X C \^ and 
Y C Vj with \Y\ > e\V{G)\/k, we have 

\x\-\Y\ m-\v,\ - ■ 

The partition V is e-regular if all but at most ek'^ pairs {Vi, Vj) are e-regular. 

Every e-regular partition is weakly (Te) -regular, but in the reverse direction only a much 
weaker implication holds: a weakly e-regular partition with k classes is e-regular with e = \Jk'^e. 

The "original" Szemeredi Lemma can be stated as follows. 
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Lemma 6.3 (Szemeredi Regularity Lemma |31| ) For every s > and I > there is a 
k{e, I) > such that every simple graph G = {V, E) with at least I nodes has an e-regular 
partition into at least I and at most k{e, I) classes. 

Let G be a graph and let V be an equitable partition of V{G). Then G/V is a weighted 
graph with almost equal nodeweights. We modify this graph by making all nodeweights equal 
to 1. This way we get a weighted graph G with nodeweights 1 and edgeweights in [0, 1]. 

For every bounded, simple graph parameter / and every weighted graph H with nodeweights 
1 and edgeweights in [0, 1], define 

f{H) = E{f{G{H)), 

where G{H) is the graph obtained by the randomizing procedure described in Section l473l Clearly 
/ is an extension of /. 

Supplement 6.4 Either one of the following conditions is also equivalent to testability: 

(f) For every e > there is an e' > and an no G Z+ such that if G is any graph with 
1^(^)1 > (I'f^d V is an equitable weakly e' -regular partition of G with uq < \V\ < e'\V{G)\, 
then\f{G)-f{G^V)\<e. 

(g) The parameter f has an extension f to (finite) weighted graphs with nodeweights 1 and 
edgeweights in [0, 1] such that 

(g.l) for every fixed n, f is a continuous function of the edgeweights on n-node graphs, and 

(g.2) for every e > there is an e' > and an no G Z+ such that if G is any graph with 
1^(^)1 > uq andV is an e' -regular partition of G with uq < \V\ < e'\V{G)\, then \f{G) — f{G^ 
V)\<e. 

Condition (g) is a priori weaker than (f ) on two counts: First, (g) allows an arbitrary extension 
of / to weighted graphs, while (f) makes assumptions about a specific extension /. Second, (g) 
states a condition about e-regular partitions, for which the condition may be easier to verify than 
for weakly e-regular partitions. Of course, we could formulate two "intermediate" conditions, in 
which only one of these relaxations is used. 

Condition (g.l) concerns graphs on a fixed finite set, so when we say that / should be 
"continuous", we do not have to specify in which metric we mean this. But to be concrete, we 
will use the du distance metric on these graphs. 

6.2 Proofs 

Before proving the equivalence of the above conditions, we state and prove a simple lemma. 
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Lemma 6.5 Let G, G' be weighted graphs with edgeweights in some interval I of length \I\ and 
total nodeweight one. If G and G' have equal edgeweights hut different nodeweights, then 

5u{G,G') <\I\Y,MG) - a,{G')\. 

i 

Proof. Let aj be the nodeweights of G, and a'^ be those of G'. Consider a couphng X such 
that Xii = mm{ai,a'j}. Then 

Sn{{G,G') < da{G[X],G'[X^]) < J2 X,kX^i\(3i,iG) - (3ki{G')\ < \I\ ^^kX.i 

i,j,k,l ij^j or 

i i i 

□ 

After these preparations, we are ready to prove the results of the last section. 

Proof of Theorem 16.11 We first prove that (a), (b), (c) and (e) are equivalent: 

(a) ^(b): The definition of testability is very similar to condition (b): it says, in this language, 
that a random set Sk on k nodes of G as in (b) satisfies 

\f{G)-f{Q{k,G))\<e 

with large probability. This clearly implies that this difference is small on average. 

(b) ^(c): Let {Gn) be a convergent sequence with oo. Given e > 0, let k be such 
that for every graph G on at least k nodes, \ f{G) — E(/(G(A;, G))! < e. Since G„ is convergent, 
t(-F, Gn) tends to a limit for all graphs F on k nodes, from which we get that tind(-^; Gn) tends 
to a limit tind(-^) for all graphs on k nodes. This means that Pr(G(/i;, G„) = F) ^ ^md(-^), and 
so 

E(/(G(fc,G„))) ^ Y.^UF)f{F) = ak. 

F 

As a consequence, we have that for all sufficiently large n, 

\f{Gn)-ak\ < \f{G)-E{f{G{k,Gn))\+e<2e, 

so f{Gn) oscillates less than As if n is large enough. This proves that the sequence {f{Gn)) is 
convergent. 

(c) ^(e): Suppose that (e) does not hold for some e > 0; then there are two sequences (G„) 
and {G'n) of graphs such that |r(G„)|, \V{G'n)\ ^ oo, fc(G„,G"J ^ 0, but \f{Gn)-f{G'^)\ > e. 
We may assume that both graph sequences are convergent; but then 5n(G„, G'n) — ^ implies that 
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the merged sequence (Gi, G'^, G2, G'2, ■ ■ ■) is also convergent, so by (c), the sequence of numbers 
(/(Ci), f{G[), /(G'2), /(G2), . . . ) is also convergent, a contradiction. 

(e) ^(a): Suppose (a) does not hold. Then there is an e > and a sequence {On) of graphs 
with |l^(G'„)| > n such that with probability at least e we have that \f{Gn) — /(G(n,G„))| > e 
for all n. Now choose > and an uq G in such a way that \ f{Gi) — /(G2)| < ^ whenever 
Gi,G2 are two graphs with |F(G'i)|, |V^(G'2)| > '^o and S\j{Gi,G2) < £0 (this is possible by (e)). 
But by Theorem 14.71 (iii), we have 5n(G„, G(A;, — > in probability, implying in particular 
that for n large enough, S\j{Gn, G{k, Gn)) < Sq with probability at least 1 — e/2. By our choice 
of this implies that for n large enough, — f{Q{k,Gn))\ < £ with probability at least 

1 — e/2, a contradiction. 

We continue with proving that (d) is equivalent to (a), (b), (c) and (e). 

(e)^(d): For W G W[o,i], define f{W) = lim^^oo where is any sequence of 

graphs such that Gn — > W and |\^(G'.„)| 00 (by (e), f(W) does not depend on the choice of 
the sequence Gn as long as G.„ — ^ W). We prove that this functional has the desired properties. 

Let e > 0, and let e' and Uq be as given by (e) with e replaced by e/3. To prove continuity, we 
will prove that |/(H^) — fiW')\ < e whenever \\W — W'\\u < e' /?>. Considering a graph sequence 
tending to W, we can choose a simple graph G such that |V(G)| > tiq, 5u{G,W) < e'/S and 
\f{G) — f{W)\ < e/3; similarly, we can choose a simple graph G' such that |V(G")| > tiq, 
Sn{G', W) < e'/3 and \f{G') - f{W')\ < e/3. Then 

6d{G, G') < 6d{G, W) + 6d{W, W) + 6n{W', G') < e\ 

and hence by (e), |/(G) - f{G')\ < e/3. But then 

\f{W) - f{W')\ < \fiW) - /(G)| + |/(G) - /(G')| + |/(G') - fiW)\ < e 

as claimed. Note that our proof shows that, in fact, / is continuous in the 6n metric. 

The fact that fiWc,,) — f{Gn) whenever |y(G'„)| 00 can now be easily verified by 
contradiction. Indeed, assume that this is not the case. Using compactness, we may choose a 
subsequence such that Gn W for some W. But this implies that f{Gn) — > fiW), and by the 
continuity of /, also that /(W^g„) fiW)^ ^ contradiction. 

(d) ^(c). Consider a convergent graph sequence (G„) with |l^(G„)| 00, and let W & W 
be its limit. Then So(Wg„, W) — > 0, and by Lemma [5731 \\Wg'^ — W\\a ^ for a relabeling of 
Gn- By the continuity of /, we have /{W^J - f{W) 0. By assumption, /((?„) - /{WcJ = 
f{Gn) - /(VTgJ ^ 0, and so /(G„) - f{W) 0. This proves that (/(Gn)) is convergent. □ 

Proof of Supplement 16. 2t 

(e) ^(e.l), (e.2), (e.3): To see that (e.l) is a special case of (e), it suffices to note that 
if Gi and G2 are two different graphs on the same set of n nodes, and d\j{G, G') < e', then 
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n > 1/^/e'. For (e.2), note that 6\j{G[m], G[m']) = for all m,m'. For (e.3), it suffices to verify 
that 6o{G,GKi) < l/\V{G)\. 

(e.l), (e.2), (e.3)^(c): Suppose that (c) does not hold. Then there exist two graph sequences 
and with oo and |V^(G'„)| oo such that Gn^G'^ W, f{Gn) a and 

fi^'n) — ^ ^ as n — i> oo, but a ^ h. 

By (e.l), there exists an e > such that if G and G' are two graphs on the same node 
set, and du{G,G') < e, then |/(G) - f\G')\ < \a - b\/A. Let si = {e/32)^\ By Lemma EH 
for every n there is a simple graph if„ whose number of nodes k depends only on e such that 
6\j{Gn, Hn) < £1/2. By selecting an appropriate subsequence, we may assume that = H is 
the same graph for all n. Since (Gn) and (G^) have the same limit, it follows that 5n(G^, H) < e\ 
for all n that are large enough. 

Let us add to each G„ at most k — \ isolated nodes so that the resulting graph G* has kvnn 
nodes for some integer m„. The m„-fold blow-up -ff [m„] of H then satisfies ^□(G* , i/[m„]) < 5i, 
and so by Theorem 12.31 for a suitable overlay of G„ and if[m„], we have du[G*n^ii^r^ < e, 
and so, by the definition of e, 



i/(G:)-/(i/K])i< 



\a — b\ 



4 

Using (e.3), we see that /(G*) — f{Gn) 0, and hence 



|/(G„)-/(i7K])| < 



\a — h\ 



3 

if n is large enough. Similarly, H has a m'„-node blow-up H[m'^ such that 

l/(G;)-/(iJK])|<^^. 

But since H[mn\ and -^[tti^] are blow-ups of the same graph if, (e.2) implies that /(if[m„]) — 
f\H[m'^) ^ 0, a contradiction. □ 

Proof of Supplement 16. 4t We will assume (without loss of generality) that |/| < 1. 

(e)^(f). Choose Sq and rio so that (i) for n > no the bound in Lemma [4.31 is at most Eq with 
probability at least 1 — e/4 and (ii) for two graphs Gi and G2 with |y(Gi)|, |\^(G2)| > riQ and 
fa(Gi,G2) < 3£o, we have |/(Gi) - /(Gs)! < e/4. Suppose that \V{G)\ > no and no < \V\ < 
eo\V{G)\. By definition of £o-regular partitions and by Lemma [631 we have 5n(G, G ^V) < 2eo. 
Furthermore, by Lemma 14. 3[ we have 5n(G -r- P,H(G ^ V)) < eo with probability at least 
1 — 5/4. If this occurs, then (5n(G, H(G ^ P)) < 3£:o, and by the choice of eo and no, it follows 
that |/(G) - /(H(G^P))| <e/2. Hence 

|/(G) - 7(G)| = I E(/(G) - /(H(G - V))) I < |(1 - ^/4) + 2(^/4) < e. 
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(f) ^(g). All we have to verify is that for weighted graphs on a fixed node set (say [n]), f is 
a continuous function of the edgeweights. Consider two weighted graphs Hi,H2 with V{Hi) = 
V{H2) = [n] and di{Hi,H2) < e/n^. By Lemma [4.2^ the randomized simple graphs G{Hi) and 
G(i?2) can be coupled so that E[(ii(G(i7i), G[H2))] < e/n^, which by Markov's inequality implies 
that G{Hi) = G{H2) with probability at least 1-e. This implies that \f{G{Hi))-f{G{H2))\ < 
e. (Note that this argument did not require that / is testable.) 

(g) ^(e.l),(e.2),(e.3). Let e > 0, and consider the extension / postulated in (g). By property 
(g.2), we can choose an ei > and an rti G Z+ such that if |V^(G')| > rii and V is an ei- 
regular partition of G with ni < \V\ < ei\V{G)l then |/(G) - f{G ^ V)\ < s/3. Using the 
Regularity Lemma [673| fix /c = k{ei/2, rii) > ni so that every graph G with at least rti nodes has 
an (ei/2)-regular partition with at least rii and at most k classes. Let = maxjrii, 

Using condition (g.l), choose an £2 > such that for any two graphs Hi and H2 on the same 
set of m < TT-o nodes with d\j{Hi,H2) < €2 we have |/(i^i) — /(-f^2)| < £^/3. Finally, choose 
e' = min{e2/4,e?/(8P)}. 

To prove (e.l), let G and G' be two simple graphs on the same node set with dci{G, G') < e'. If 
n= \ V{G)\ < no, then 1/(^1) - /(G'2)| = |/(Gi) - /(G'2)| <e/3, so we can assume \ V{G)\ > uq. 
Let V = {Vi, . . . , V;) be an (£:i/2)-regular partition of G and ni < I < k. From the assumption 
that dci{G,G') < e' < e'^/(8/c^), it follows that V is an ei-regular partition of G'. By the choice 
of El and rii it follows that 

|/(G) - /(G - ^)l < I and |/(G0 - /(G' - V)\ < |. 

Furthermore, we have 

n 



.. 2 

n^dn{G, G') > l^daiG ^V,G' ^ V) 



I 



and so 



n 

Hence by the choice of 621 we have 



2 ^ -2 



n 



dn{G ^V,G'-rV)<— - du{G, G') < As' < 62. 

I 



|/(G-P)-/(G'-P)|<|. 

Summing up, 

|/(G) - f{G')\ < |/(G) - f\G - V)\ + |/(G - P) - f\G' - V)\ + \f{G' - - f{G')\ < e. 

The proof of (e.2) is similar, but divisibility concerns cause some complications. We have to 
show that f{G[q\) is a Cauchy sequence, i.e., we have to show that given G and 5 > 0, we can 
find a go such that |/(G[g]) — /(G[g'])| < 2e whenever g, g' > go- Let n = |V^(G)|, p > 1 large 
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enough, and let V = {Vi, . . . , Vi} be an (£:i/2)-regular partition of G[p] with rii < I < k. By our 
choice of p we have, in particular, that / < y^P^ "^^ ^PP^J (g-2) to get 

\f{G[p])-f{G[p]^V)\<'-. (6.1) 

Let r be sufficiently large, and let g = + s with < s < p. The graph G[pr] arises from 
G[p] by blowing up each node into r nodes, and then G[q] arises from G[pr] adding sn further 
nodes. 

First we consider the graph G[pr]. The partition V determines a partition Q = {Ui, . . . , Ui} 
oiV{G[pr]). Let 

_ eG\p]iVuVj) _ eG[pr]{Ui,Uj) 

denote the edge density between Vi and Vj in G[p] (which is the same as the edge density between 
Ui and Uj in G[pr]). 

We claim that if {Vi, Vj) is an (ei/2)-regular pair in V, then for all X C Ui and Y C Uj with 

\Y\ > 2einpr / (31), we have 

<^^;^^<p«+|. (6.2) 

For u & Vi, let x'„ denote the number of elements in X among the r copies of u in G'[pr],and let 
Xu = x'^/r. Clearly < x„ < 1 and XIm^m — [^eiupr / {3l)\. We define for v & Vj analogously. 
Then we have 

eG[pr]{X,Y) - \X\ ■ \Y\{pij - El) = ^ ^r'^XuVviauv - Pij - ^). (6.3) 

Since the right hand side is linear in each Xu, it attains its minimum over < x„ < 1, XIm^m — 
\_2einpr / {3l)\, at a vertex of this domain, which is a 0-1 vector. Similarly, the minimizing 
choice of the is a 0-1 vector. Let S = {u : = 1} and T = {v : yv = 1}, then 
\S\, \T\ > L2einpr/(3/)J > ^-\V{G\p]\/l. The right hand side of (El is equal to 



^2 



(eaH(^,T)-|5|-|T|(p,, -|)) >0, 



since {yi,Vj) is (£:i/2)-regular. This proves the first inequality in (16. 2p : the proof of the second 
is similar. 

Now Q is not necessarily an equitable partition; the largest and smallest class sizes may differ 
by r, not by 1. Let us remove r nodes from those classes of Q that are too big. To get a partition 
of \^(G[g]), we have to add back these nodes {t < I) and sn further nodes. Let us distribute these 
nodes as equally as possible between the classes, to get an equitable partition Q! = {U[, . . . , [//} 
oiG[q]. 
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We claim that Q! is ei-regular. Consider a pair (V^, V,) that is (£:i/2)-regular in G\p\, and 
subsets X' C U[ and Y' C f/j with |X|, |y| > einq/l. Remove the new nodes from X' and Y' to 
get X C f/i and y C Uj. Clearly |X|, |r| > Siuq/l - r - [sn//] > 2£irapr/(3/), and so is 
satisfied. Now it is easy to check that 



m\-\u;\ 



\u,\ ■ \u, 



< 



and 



eG[,](X',r) eGipr]{X,Y) 



\X'\ ■ \Y' 



\X\ ■ \Y\ 



< 



if p > 32l/el and q > 32p/el, which proves that 



eGi,iX'X)\X'\.\Y'\ 



<ei. 



Thus Q' is an ei-regular partition of G[q]. It follows by (g.2) and the choice of ei that 

\f{G[q])-f{G[q]^Q')\<l. 



(6.4) 



The weighted graphs G[p] -rV and G[q] -r- Q' are very close to each other. In fact, G[p] ^ V = 
G[pr] Q, while it is easy to check that da{G[pr] -i- V, G[q] Q) < 62 if q > 4:p/e2, and so (g.l) 
implies that 

|/(G[p]-P)-/(G[g]-QOI<|- (6.5) 

Now (O), (EH) and ([63]) imply that |/(G[p]) - /(G[g])| < £■ So if q,q' > max{32p/el Ap/€2}, 
then 

\f{G[q]) - f{G[q'])\ < \f{G[q]) - f{G[p])\ + \f{G[p]) - f{G[q'])\ < 2e. 

Thus f{G[q]) is a Cauchy sequence, which proves (e.2). 

The proof of (e.3) is similar but easier, and is left to the reader. □ 



7 Concluding Remarks 

7.1 Norms Related to the Cut-Norm 

The cut-norm (and the cut-distance of graphs) is closely related to several other norms that are 
often used. We formulate these connections for the case of graphons, but similar remarks would 
apply to the d\j distance of graphs. 



We start with the remark that we could restrict the sets S, T in Definition l3.3t it turns out that 
"reasonable" restrictions only change the supremum value by a constant factor. In particular, it 
is not hard to see that 



-\\W\\n < sup 

^ S=T 



W 



SxT 



< \\w\ 



□ < sup 

S'nT=0 



W 



SxT 



< \\w\ 



D, 



(7.1) 
(7.2) 
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and 



2 

3 



sup 


/ w 


< sup 


/ W 


< sup 


/ W 


5nT=0 


J SxT 


5=[0,1]\T 


J SxT 


S'nT=0 


J SxT 



(7.3) 



see appendix for details. 

To relate the cut norm to homomorphisms, we start with noticing that 



(7.4) 



and hence the functional t(C4, .)^/^ defines a norm, called the trace norm or Schatten norm, on 
W. (More generally, any even cycle leads to a norm in this way.) The following lemma shows 
that the norm in (17.41) is intimately related to the cut-norm. 



Lemma 7.1 For f/ G W with 



< 1, we have 



^t{C„U)<\\U\\a<t{C„UY/\ 



Proof. The first inequality is a special case of (14.271) . To prove the second inequality, we use 
dSSD and dZai): 



oo^i = sup if, Tug), 

I/I,l9l<l 



and here 



2-{Tug,Tugy/' 

.2 111/ 
U\\2 



< Wfh ■ hh ■ \\Td\\'£2 < r^llf = (Tr T^y/' = t{C,, Uf/'. 



□ 



Lemma [TD allows for a significant simplification of the proof that the sample from a weighted 
graphs is close to the original graph. Indeed, it is possible to use the following easy lemma instead 
of the quite difficult Theorem 14.61 (or the equally difficult results of [3]) to establish a slightly 
weaker version of Theorem 14.71 which is still strong enough to prove the equivalence of left- 
convergence and convergence in metric (see [15] for details). 



Lemma 7.2 Let Q < e < I, Q < 5 < I, and let U eW, 



<1. // 



then 



\\U\\n<\e"' and ^^>^ln(^) 
p(\\il{k,U)\\u<e^ > 1-5. 
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Proof. Applying first Lemma [7. II and then Lemma [4.41 we have 

||H(fc, U)\\d< tic, U{k, U)f/^ < (t(C4, U) + ^) 
with probabihty at least 1 — 5. Thus, using Lemma 17.11 again, we get 

mk.U)\\u<[t{C,,U) + -) < [a\\U\\u+-) <e, 

as claimed. □ 

A substantial advantage of the norm t(C4, W) over the cut-norm is that for weighted graphs 
(and for many other types of graphons W, for example, for polynomials), it is polynomial-time 
computable. A better polynomial-time computable approximation is the Grothendieck norm, 
which approximates the cut-norm within a constant factor (see [5]). 

7.2 A Common Generalization of Lemma 12.41 and Lemma 13.31 

As stated. Lemma 13.31 does not generalize Lemma 12.41 since the partition in Lemma 13.31 is not 
necessarily aligned with the steps of the stepfunction Wg- But the following lemma implies both 
Lemma [2.41 and Lemma [3. 3[ 

Lemma 7.3 Let A be an algebra of measurable subsets of [0, 1], and let 

W . 



|W^IU,n= sup 

S,T£A 



SxT 



Then for every graphon W and every e > 0, there exists a partition V of [0, 1] into sets in A 
with at most 4!^^/^^^"^ classes such that 

\\W - WvlUn < £\\W\\2. 

Proof. Let P of be a partition of [0, 1] into q classes in A, let S,T & A, and let V be the 
partition generated by S, T and V. Clearly V' has at most 4g classes, all of which lie in A. Since 
Wp' gives the best L2-approximation of W among all step functions with steps V', we conclude 
that for every real number t, we have 



Bounding the right hand side by || W^-iyp||2-2t(l5xT, W-W-p)+t'^ and choosing t = {Isxt, W- 
W-p), this gives 

{lsxT,W -Wvf < \\W-Wv\\l - \\W -Wp,\\l = \\Wv'\\l - \\Wr\\l 
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Taking the supremum over all sets S,T G A, this gives 

\\W - WAao< sup WWv'Wl - WWvWl (7.5) 
v 

where the supremum goes over all partitions of [0, 1] into 4g classes in A. From this bound, the 
lemma then follows by standard arguments. □ 



7.3 Right Convergence 

When studying homomorphisms from G into a small graph H it will be convenient to consider 
graphs H with nodeweights, ai{H) > 0, and edgeweights Pij{H) G M (with i running over all 
nodes in H, and ij running over all edges of F). For such a graph, we define 

hom{G,H) = n «'^«(^) n %)>0(i)(^) 

where the sum runs over all maps (j) from V{G) to V{H) and (3^^i{H) is set to zero if 00' is not 
an edge in H . We call hom(G, H) the weighted number of //-colorings of G. 

For dense graphs G, the weighted if-coloring numbers hom(G, H) turn out to be most in- 
teresting when all edgeweights of H are strictly positive (we say that if is a soft-core graph 
if this is the case). Under this assumption, the homomorphism numbers hom(G', if) typically 
grow exponentially in the number of edges in G. For homomorphism into small graphs H , it is 
therefore natural to consider the logarithm of hom(G, H) divided by the number of nodes in G 
to the power two. We will also consider the "microcanonical ensemble", where the number of 
nodes in V{G) colored by a given color c G V{H) is fixed to be some constant proportion Cc of 
all nodes. We denote these homomorphism densities by homa(G', H), where a is the vector with 
components c G V{H). 

The above perspective leads to two a priori different notions of convergence: A sequence 
of weighted graphs (Gn) will be called convergent from the left if the homomorphism densities 
t{F, Gn) converge for all finite graphs F, and it will be called convergent from the right if the 
quantity |y(G„)|~^ log homa(G'ri, if) converges for all a and all soft-core graphs H. It will turn 
out that these two notions are closely related. Convergence from the left implies convergence 
from the right (both for the standard homomorphism numbers and the microcanonical ones), and 
convergence from the right for the microcanonical homomorphism numbers implies convergence 
from the left. This will be discussed in more detail in the continuation of this paper [16]. 
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8 Appendix 



In this appendix, we collect various proofs which were omitted in the body of the paper. 
8.1 Proof of Corollary [331 

Proof of (i): Let V be a partition of [0, 1] into q' < 2^^/^^'^") classes such that \\W - Wv'\\u < 
y||H^||2. Then there exists an equipartition V of [0,1] into q classes such that at most q' 
of its classes intersect more than one class of V. Let R be the union of these exceptional 
classes, and let U be the step function which is equal to W-p' on ([0,1] \ R)"^, and on the 
complement. Then — U\\a can easily be bounded by decomposing the sets S,T in the 
definition of the cut-norm into the part contained in [0, 1] \ -R and its complement. Using the 
fact that X{R) <j< 2-79/(8£2) < ^22-79/8^ ^Yiis leads to the estimate 

\\W - U\\n < (y + V^SACR)) ||1^||2 < + Vs ■ 2^79/8 j \\w\\2 < '-\\W\\2. 

By construction, t/ is a step function with steps in V. Using the bound (13.111) . we conclude 
that \\W — W-p||n < 2||W^ — f/||n < £:||W^||2, which gives the first statement of the corollary. The 
second assertion in statement (i) is proved in a similar way, starting from a common refinement 
of V and P. 

Proof of (ii): Let V = V{G), let n = \V\, and assume without loss of generality that 
ac = 1. Choosing a partition V' = (y(, . . . ,V^,) of V with q' < 2^^/'^^'^^) classes such that 
da{G,G-p') < ^\\G\\2, we would like to divide each class in V' into subclasses Vi such that all 
of them obey the condition (13.121) . To this end, we proceed as follows: Starting with V-[, we 
successively remove sets Vi, V2, . . . from first V{, then V2, etc. so that 

1 



and after each step 

t 



< amax(G') for alH = 1, . . . , g 



1=1 



^ Qmax(G') 



When it is not possible to further remove a set Vi from V( while maintaining these constraints, 
we are left with a (possibly empty) remainder Ri that has weight aG[Ri] < ^/q (otherwise, we 
could have continued for at least one more step). Continuing with Vg, . . . ,V^,, we will eventually 
end up with q — q' < t < q disjoint sets Vi, . . . ,Vt obeying the condition (18. ip . and r < q' non- 
empty remainders Ri such that the total weight of their union, R = [J^Ri, obeys the condition 
\c(G[R] — {q — t)/q\ < amax/2. Using this condition, it is not hard to see that R can be split into 
q — t final sets Vf+i, ■ ■ ■ ,Vg obeying the condition (18.11) . Since each of the remainders had weight 
at most 1/q, the total weight of R is at most q'/q < 2''^^^^^^^^ < 6^2''^^/^. 

From here on the proof is completely analogous to the proof of (i). □ 
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8.2 Proof of Lemma 12.51 

We start with the observation that the proof of the last section actually gives the stronger bound 

W^pIId < e(^^ + Vs ■ 2-'^^/^^\\W\\2 < 0.982e||M/||. 
Applying this bound to Wq, this gives a weighted graph H = Wg/V on g > 2^°/"^^ > 2^'^/e^ 



nodes such that H has nodeweights one and 5n(G', H) < 0.982£:. Combined with Lemma I4.3[ 
this gives the existence of a weighted graph H on [q] such that 

4 

SoiG, H) < 0.982e + — < e, 
as required. □ 

8.3 Proof of Lemma 13.51 

We notice that 5n(t/, W) as well as the infima and limits in (13.141) . (13.151) and (13.161) are continuous 
in both U and with respect to the ||.||n norm. This fact and Lemma 13.21 imply that it is 
enough to prove the lemma for graphons U and W that are interval step functions with equal 
steps, corresponding to some finite graphs G and G' . 
Furthermore, the inequalities 

inf ||[/*- VT^IIn <inf VT^'lln < liminf min ||f/ - VT^Hn 

0,1/; 1p n— >00 TT 

are trivial, so it suffices to prove that 

5u{U,W) < \\U^-W^u (8.2) 
for all measure-preserving maps 0,-1^ : [0, 1] [0, 1], and 

limsupmin||t/- VT'lIn < 5u{U,W), (8.3) 

n— >oo '"^ 

where the infimum is over all permutations of [n]. 

To prove (18. 2p . we consider two weighted graphs G and G' with ac = olq = 1, together 
with the step functions U = Wq and W = Wq'- Let /i,...,J„ be the intervals [0, «!((?)], 
(ai(G), ai{G) + 02 (C)], . . . , {ai{G) + ■ ■ ■ + a„_i(G), 1], and similarly for /{,..., 1'^,. For two 
measure-preserving maps (p, ip : [0, 1] [0, 1], we then rewrite the norm on the right hand side 
of (18. 2p in the explicit form 

\\U^-W^u= sup / {WG{m.'P{y))-WG'{ij{x),'iP{y)))dxdy 

S,TC[0,1] JsxT 
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The supremum on the right hand side is attained when S and T are unions of sets of the form 
Viu = (f'^^ih) n ip~^{I[X i e V{G), u e V{G'). But for these sets, the integral on the right is a 
sum of terms of the form 

f {WG{(j){x),<P{y)) - WG'Wx),ilj{y))) dxdy = A,(G)/3™(G")X,„X,„ 

where Xiu is the Lebesgue measure of the set Viu- As a consequence, we have that 

\\U'''-W^\\a = daiG[X],G'[X^]). 

Using the fact that and ip are measure-preserving, it is not hard to check that X is a couphng 
of the distributions {(yi{G))i^viG) and (ttu(G"))wev^{G")5 implying that 

\\U^ - W^Wn = daiG[X],G'[X^]) > 6niU,W). (8.4) 

To show (18.31) . let us consider a coupling X G X{G,G'). First construct a special measure- 
preserving bijection : [0, 1] [0, 1] such that \\Wg - W^,\\n = dniG[X],G'[X^]). As shown 
above, this is equivalent to finding a measure-preserving map such that Xiu is the Lebesgue 
measure of fl ?/'~^(/^). But the construction of such a map is straightforward. Indeed, for 
iu e V{G) X V{G'), let k = ai{G) + ■■■ + ai_i(G), b'^ = ai{G') + . . . ,a„_i(G"), Q„ = h + 

Xii + Xi2 + h and c'j„ = b'^ + Xiu + ^2m H h Xnu- Let /j„ and be the intervals 

liu = {ciu, + Xiu] and I^^ = (c^„, c'j„ + Xj^]. We then choose ip to be the translation that maps 
liu into Then ^"^(/^ = Ui^~H-^L) = Ui-^m and Jj n ^p^^il'u) = liu, implying in particular 
that this set has measure Xiu, as required. 

So we have two partitions {/i, . . . , 7^} and { Ji, . . . , Jm} of [0, 1] into intervals, and ip maps 
each Ik onto a J/(fc) by translation. Furthermore, we also know that both U and W are constant 
on each rectangle x Ii as well as on J^. x J/. 

Let X be a large integer, and consider the partition {Li, . . . ,Ln} of [0, 1] into intervals of 
size 1/N. We define a permutation vr of [N]. For every k < m, the intervals Ik and Jf{k) have 
the same length, and so the numbers of intervals Li contained in them can differ by at most 
one. Let it match the indices i such that Li C with the indices j such that Lj C J/(fc), with 
at most one exception. This way 7r{i) is defined for at least N — 3m integers i G [N]. Call the 
corresponding intervals Ki well-matched. We extend vr to a permutation of [N] arbitrarily. 

We see that W^{x,y) = iy^(x,y) whenever both x and y belong to well-matched intervals. 
Hence 

\\W^ - W^a < \\W^ - W% < —\\W\U 

and so 

\\U - W^Wa <\\U -W^\\a+\\W^ - W^\\a < Sa{U, W) + —\\W\U 
This implies (El). □ 
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8.4 Proof of Lemma 14.41 

Starting with the proof of (14. lip , let us assume that k'^/n < £:/(lllog2) (otherwise the bound 
(14. lip is trivial). Using the bound (14. 9p . we then estimate the probability on the left hand side 
of fl4AT|) by 



t(F,H(n,iy))-E[t(F,H(n,l^))] 



> \e 

n 



< P 



t{F,Yl{n,W))-E[t{F,Yl{n,W))] 



> e{ 1 



lllog2 



Applying Lemma [4. II (i) and the observation that t{F, W[{zi, . . . , Zn}]) changes by at most 2k /n 
if we change one of the variables Zi we immediately obtain the bound (14. lip . 

Expressing G{n, W) as a function of the random variables Zi, . . . , Z„ introduced above, and 
observing that t{F, G(n, W)) changes by at most k/n if we change one of the variables Zi, the 
proof of (14.120 is virtually identical to that of (14. lip . We leave the details to the reader. □ 

8.5 Detail Concerning Example 15.11 

Let H = Knm,nm and G = K^^n be the graphs considered in Example 15. 1[ Here we show that 
for every n, there exists an m such that with large probability, \\Wh — W^g'IId > 1/10 every 
relabeling G" of G. 

Indeed, let S' and T' be the color classes of G', and let / and J be the subsets of [0, 1] 
corresponding to S' and T'. Let X and Y be the subsets of V{H) that correspond to / and J in 
Wh. Then 



\WH-WG'\\n> I [Wg'-Wh) 

JlxJ 



eG'{S,T) eH{X,Y) _1 eH{X,Y) 



Now because of the random labeling of V{H), the expectation of the second term is 1/8, and 
the value of the second term will be arbitrarily highly concentrated around this value if m is 
sufficiently large. There are only n! possible relabelings of G, so with large probability the second 
term will be less than 3/20 for all of these, which proves our claim. 



8.6 Proof of (O), (O) and (O) 



The upper bound in each of the three equations is trivial, so we only need to prove the lower 
bounds. To prove these, we introduce the notation 



W{S,T)= [ W{xi,X2)dxidx2 

J SxT 



for the value of the "cut between S and T'' 
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Proof of the lower bound in ( 17. ip : This follows easily by inclusion-exclusion applied to 
W{SUT,SUT). 

Proof of the lower bound in 07.21) : To this end, we first approximate W hj a. weighted 
finite graph without loops. Next, we discard each point in \ T and T\S with probability 1/2, 
and then we add each point in SCiT uniformly at random either to the remaining points in S\T, 
or to the remaining points in T \ S*. In expectation, the weighted number of edges between the 
resulting, disjoint sets is off by a factor of 1/4, giving the first bound in (17.21) . 

Proof of the lower bound in (17. 3p : To prove this bound, we consider two disjoint sets 
S and T and their complement R = [0, 1] \ (S* U T). We then express each of the three cuts 
W{S,T U R), W{T,SU R), and W{S U T,R), in terms of W{S,T), W{S,R) and W{T,R). 
Combining these three relations leads to the desired bound in (17. 3p . 
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